198 lines
36 KiB
Plaintext

2025-12-22 04:28:50.726 DEBUG [tests.conftest] Running fixture setup: test_id
2025-12-22 04:28:50.726 DEBUG [tests.conftest] Running test: test_relay_get_message_while_one_peer_is_paused with id: 2025-12-22_04-28-50__ff6c0b6f-e075-499b-856f-78e96eb1591f
2025-12-22 04:28:50.726 DEBUG [src.steps.common] Running fixture setup: common_setup
2025-12-22 04:28:50.726 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2025-12-22 04:28:50.727 DEBUG [src.steps.relay] Running fixture setup: setup_main_relay_nodes
2025-12-22 04:28:50.734 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-22 04:28:50.734 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2025-12-22_04-28-50__ff6c0b6f-e075-499b-856f-78e96eb1591f__wakuorg_nwaku:latest.log
2025-12-22 04:28:50.734 DEBUG [src.node.waku_node] Starting Node...
2025-12-22 04:28:50.734 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-22 04:28:50.736 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-22 04:28:50.736 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.18.201
2025-12-22 04:28:50.736 DEBUG [src.node.docker_mananger] Generated ports ['39348', '39349', '39350', '39351', '39352']
2025-12-22 04:28:50.736 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-22 04:28:50.736 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-22 04:28:50.736 DEBUG [src.node.waku_node] Using volumes []
2025-12-22 04:28:50.737 DEBUG [src.node.docker_mananger] docker run -i -t -p 39348:39348 -p 39349:39349 -p 39350:39350 -p 39351:39351 -p 39352:39352 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=39350 --rest-port=39348 --tcp-port=39349 --discv5-udp-port=39351 --rest-address=0.0.0.0 --nat=extip:172.18.18.201 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=d6fbe205dfeb1ebca0ecc33accdca0a0cb5fbff973e80ab02d469acdd167c138 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=39352 --metrics-logging=true --relay=true
2025-12-22 04:28:50.949 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.18.201 waku 7705bfdbf5a88f42a7913add4e9fd356b1935f8087a6703527d105a58fbb0382
2025-12-22 04:28:50.989 DEBUG [src.node.docker_mananger] Container started with ID 7705bfdbf5a8. Setting up logs at ./log/docker/node1_2025-12-22_04-28-50__ff6c0b6f-e075-499b-856f-78e96eb1591f__wakuorg_nwaku:latest.log
2025-12-22 04:28:50.989 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 39348
2025-12-22 04:28:50.990 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-22 04:28:51.990 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39348/health" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:51.994 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-22 04:28:51.994 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-22 04:28:51.994 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39348/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:51.996 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.18.201/tcp/39349/p2p/16Uiu2HAmUcYqMivU6175rMmmrTKojTzHjpB9dLYcKeyKL73NRzDM","/ip4/172.18.18.201/tcp/39350/ws/p2p/16Uiu2HAmUcYqMivU6175rMmmrTKojTzHjpB9dLYcKeyKL73NRzDM"],"enrUri":"enr:-L24QGLmC8XCOvhm9Pjvtufl_7BF2fy3dUu4rvCkuYBSI1FJEft9Ox9tYB7z1SQ9dcMgW3gSS40nclSReUGbw7OK3C8CgmlkgnY0gmlwhKwSEsmKbXVsdGlhZGRyc5YACASsEhLJBpm1AAoErBISyQaZtt0DgnJzhQADAQAAiXNlY3AyNTZrMaED7Rr6ZI12EJeArhSmBsfvcHlXvCG7vcLPaI93DTJ8ZCCDdGNwgpm1g3VkcIKZt4V3YWt1MgE"}'
2025-12-22 04:28:51.997 INFO [src.node.waku_node] REST service is ready !!
2025-12-22 04:28:52.003 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-22 04:28:52.004 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2025-12-22_04-28-50__ff6c0b6f-e075-499b-856f-78e96eb1591f__wakuorg_nwaku:latest.log
2025-12-22 04:28:52.004 DEBUG [src.node.waku_node] Starting Node...
2025-12-22 04:28:52.004 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-22 04:28:52.005 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-22 04:28:52.006 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.250.142
2025-12-22 04:28:52.006 DEBUG [src.node.docker_mananger] Generated ports ['59673', '59674', '59675', '59676', '59677']
2025-12-22 04:28:52.006 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-22 04:28:52.006 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-22 04:28:52.006 DEBUG [src.node.waku_node] Using volumes []
2025-12-22 04:28:52.006 DEBUG [src.node.docker_mananger] docker run -i -t -p 59673:59673 -p 59674:59674 -p 59675:59675 -p 59676:59676 -p 59677:59677 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=59675 --rest-port=59673 --tcp-port=59674 --discv5-udp-port=59676 --rest-address=0.0.0.0 --nat=extip:172.18.250.142 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=ede47e27d0330af621f0009ddd7a9efdf505d4ec8eba2bb93dd3d4df9cd5d2d3 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=59677 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QGLmC8XCOvhm9Pjvtufl_7BF2fy3dUu4rvCkuYBSI1FJEft9Ox9tYB7z1SQ9dcMgW3gSS40nclSReUGbw7OK3C8CgmlkgnY0gmlwhKwSEsmKbXVsdGlhZGRyc5YACASsEhLJBpm1AAoErBISyQaZtt0DgnJzhQADAQAAiXNlY3AyNTZrMaED7Rr6ZI12EJeArhSmBsfvcHlXvCG7vcLPaI93DTJ8ZCCDdGNwgpm1g3VkcIKZt4V3YWt1MgE
2025-12-22 04:28:52.206 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.250.142 waku cde77687c71c4c97366946fab052547b21843f75011a4ef6f17555de490d347e
2025-12-22 04:28:52.237 DEBUG [src.node.docker_mananger] Container started with ID cde77687c71c. Setting up logs at ./log/docker/node2_2025-12-22_04-28-50__ff6c0b6f-e075-499b-856f-78e96eb1591f__wakuorg_nwaku:latest.log
2025-12-22 04:28:52.238 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 59673
2025-12-22 04:28:52.238 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-22 04:28:53.238 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59673/health" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:53.255 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"READY"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-22 04:28:53.256 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-22 04:28:53.257 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59673/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:53.261 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.250.142/tcp/59674/p2p/16Uiu2HAm86bZup6L3o7ufeBehGKrnPkL74Tr3KAZHZ1veSN6PxKz","/ip4/172.18.250.142/tcp/59675/ws/p2p/16Uiu2HAm86bZup6L3o7ufeBehGKrnPkL74Tr3KAZHZ1veSN6PxKz"],"enrUri":"enr:-L24QLQ7ICMQo9YaBImrlL0JXPTZL3pLMUTrA_JLFf2p1lH4SeWgw5BX89l_z0G8xf3iWR_uconhdnC4GTn-jlsBe1sCgmlkgnY0gmlwhKwS-o6KbXVsdGlhZGRyc5YACASsEvqOBukaAAoErBL6jgbpG90DgnJzhQADAQAAiXNlY3AyNTZrMaECvEToAbG8RaYajJ7xTUfJnm-UAvHtG0EFsiIoJG9a86mDdGNwgukag3VkcILpHIV3YWt1MgE"}'
2025-12-22 04:28:53.261 INFO [src.node.waku_node] REST service is ready !!
2025-12-22 04:28:53.262 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:59673/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.18.201/tcp/39349/p2p/16Uiu2HAmUcYqMivU6175rMmmrTKojTzHjpB9dLYcKeyKL73NRzDM"]'
2025-12-22 04:28:53.265 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:53.266 DEBUG [src.steps.relay] Running fixture setup: setup_optional_relay_nodes
2025-12-22 04:28:53.273 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-22 04:28:53.273 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2025-12-22_04-28-50__ff6c0b6f-e075-499b-856f-78e96eb1591f__wakuorg_nwaku:latest.log
2025-12-22 04:28:53.274 DEBUG [src.node.waku_node] Starting Node...
2025-12-22 04:28:53.274 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-22 04:28:53.276 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-22 04:28:53.276 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.18.38
2025-12-22 04:28:53.276 DEBUG [src.node.docker_mananger] Generated ports ['31130', '31131', '31132', '31133', '31134']
2025-12-22 04:28:53.276 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-22 04:28:53.276 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-22 04:28:53.277 DEBUG [src.node.waku_node] Using volumes []
2025-12-22 04:28:53.277 DEBUG [src.node.docker_mananger] docker run -i -t -p 31130:31130 -p 31131:31131 -p 31132:31132 -p 31133:31133 -p 31134:31134 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=31132 --rest-port=31130 --tcp-port=31131 --discv5-udp-port=31133 --rest-address=0.0.0.0 --nat=extip:172.18.18.38 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=6b8ac2d752ff7af784b1ba84be4d859e4eaf1caeff9edbaba4399afd67fa5e89 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=31134 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QGLmC8XCOvhm9Pjvtufl_7BF2fy3dUu4rvCkuYBSI1FJEft9Ox9tYB7z1SQ9dcMgW3gSS40nclSReUGbw7OK3C8CgmlkgnY0gmlwhKwSEsmKbXVsdGlhZGRyc5YACASsEhLJBpm1AAoErBISyQaZtt0DgnJzhQADAQAAiXNlY3AyNTZrMaED7Rr6ZI12EJeArhSmBsfvcHlXvCG7vcLPaI93DTJ8ZCCDdGNwgpm1g3VkcIKZt4V3YWt1MgE
2025-12-22 04:28:53.487 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.18.38 waku a390dbe17018bc03f17a089f35ef729f6dd5f2afbb26fc617a558cb7cf33c8dc
2025-12-22 04:28:53.519 DEBUG [src.node.docker_mananger] Container started with ID a390dbe17018. Setting up logs at ./log/docker/node3_2025-12-22_04-28-50__ff6c0b6f-e075-499b-856f-78e96eb1591f__wakuorg_nwaku:latest.log
2025-12-22 04:28:53.520 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 31130
2025-12-22 04:28:53.520 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-22 04:28:54.520 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:31130/health" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:54.533 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-22 04:28:54.535 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-22 04:28:54.536 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:31130/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:54.540 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.18.38/tcp/31131/p2p/16Uiu2HAmGRbqU9t8u9uu3KQhLp9u4qGcEpE1Xsq98DYheMjNeVhw","/ip4/172.18.18.38/tcp/31132/ws/p2p/16Uiu2HAmGRbqU9t8u9uu3KQhLp9u4qGcEpE1Xsq98DYheMjNeVhw"],"enrUri":"enr:-L24QIZ9R0TdQtMlVC3jrMFDOOcOgJyHVkhNDyRlL3TvNlzFXMSP9CgQrmPLfA99mrJ432hlJFSDyHTqGsQyyVFtd4cCgmlkgnY0gmlwhKwSEiaKbXVsdGlhZGRyc5YACASsEhImBnmbAAoErBISJgZ5nN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDOAC9h8UD6A-q-Ph_dOCYbOAX_QwUPd6ckpxdDWmh9w6DdGNwgnmbg3VkcIJ5nYV3YWt1MgE"}'
2025-12-22 04:28:54.540 INFO [src.node.waku_node] REST service is ready !!
2025-12-22 04:28:54.541 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:31130/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.18.201/tcp/39349/p2p/16Uiu2HAmUcYqMivU6175rMmmrTKojTzHjpB9dLYcKeyKL73NRzDM"]'
2025-12-22 04:28:54.544 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:54.550 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-22 04:28:54.551 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2025-12-22_04-28-50__ff6c0b6f-e075-499b-856f-78e96eb1591f__wakuorg_nwaku:latest.log
2025-12-22 04:28:54.551 DEBUG [src.node.waku_node] Starting Node...
2025-12-22 04:28:54.551 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-22 04:28:54.553 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-22 04:28:54.553 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.143.76
2025-12-22 04:28:54.553 DEBUG [src.node.docker_mananger] Generated ports ['62432', '62433', '62434', '62435', '62436']
2025-12-22 04:28:54.553 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-22 04:28:54.553 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-22 04:28:54.553 DEBUG [src.node.waku_node] Using volumes []
2025-12-22 04:28:54.553 DEBUG [src.node.docker_mananger] docker run -i -t -p 62432:62432 -p 62433:62433 -p 62434:62434 -p 62435:62435 -p 62436:62436 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=62434 --rest-port=62432 --tcp-port=62433 --discv5-udp-port=62435 --rest-address=0.0.0.0 --nat=extip:172.18.143.76 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9d6f13fdd8aa05a4ebb1cd058c1ff1e9b95cbdda21c1bdddcdfaa03ea1788f3d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=62436 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QGLmC8XCOvhm9Pjvtufl_7BF2fy3dUu4rvCkuYBSI1FJEft9Ox9tYB7z1SQ9dcMgW3gSS40nclSReUGbw7OK3C8CgmlkgnY0gmlwhKwSEsmKbXVsdGlhZGRyc5YACASsEhLJBpm1AAoErBISyQaZtt0DgnJzhQADAQAAiXNlY3AyNTZrMaED7Rr6ZI12EJeArhSmBsfvcHlXvCG7vcLPaI93DTJ8ZCCDdGNwgpm1g3VkcIKZt4V3YWt1MgE
2025-12-22 04:28:54.753 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.143.76 waku 68f8cc864c6606a0f53cbcba7801b94fd70ab466df0e518bbabc78a88190cf54
2025-12-22 04:28:54.787 DEBUG [src.node.docker_mananger] Container started with ID 68f8cc864c66. Setting up logs at ./log/docker/node4_2025-12-22_04-28-50__ff6c0b6f-e075-499b-856f-78e96eb1591f__wakuorg_nwaku:latest.log
2025-12-22 04:28:54.788 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 62432
2025-12-22 04:28:54.788 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-22 04:28:55.789 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:62432/health" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:55.797 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"READY"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-22 04:28:55.798 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-22 04:28:55.799 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:62432/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:55.803 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.143.76/tcp/62433/p2p/16Uiu2HAmTYwAPr44NKg3DfpvjpDWsgKWwqeyFNNUc9U1DedV1VsM","/ip4/172.18.143.76/tcp/62434/ws/p2p/16Uiu2HAmTYwAPr44NKg3DfpvjpDWsgKWwqeyFNNUc9U1DedV1VsM"],"enrUri":"enr:-L24QJluHk5-rDHDoBgwVyJzK1qHF9gKNydge6slryq45_v0VL6MuUnsI96PnGRMEm_YFh2yNSW4F2-pfPIAwBQPfvICgmlkgnY0gmlwhKwSj0yKbXVsdGlhZGRyc5YACASsEo9MBvPhAAoErBKPTAbz4t0DgnJzhQADAQAAiXNlY3AyNTZrMaED3VI26273axIUg0sMC6FRDjdU4bMmssc-v-oJDApo_tiDdGNwgvPhg3VkcILz44V3YWt1MgE"}'
2025-12-22 04:28:55.804 INFO [src.node.waku_node] REST service is ready !!
2025-12-22 04:28:55.804 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:62432/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.18.201/tcp/39349/p2p/16Uiu2HAmUcYqMivU6175rMmmrTKojTzHjpB9dLYcKeyKL73NRzDM"]'
2025-12-22 04:28:55.808 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:55.815 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-22 04:28:55.815 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2025-12-22_04-28-50__ff6c0b6f-e075-499b-856f-78e96eb1591f__wakuorg_nwaku:latest.log
2025-12-22 04:28:55.815 DEBUG [src.node.waku_node] Starting Node...
2025-12-22 04:28:55.815 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-22 04:28:55.817 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-22 04:28:55.817 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.144.233
2025-12-22 04:28:55.817 DEBUG [src.node.docker_mananger] Generated ports ['35624', '35625', '35626', '35627', '35628']
2025-12-22 04:28:55.818 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-22 04:28:55.818 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-22 04:28:55.818 DEBUG [src.node.waku_node] Using volumes []
2025-12-22 04:28:55.818 DEBUG [src.node.docker_mananger] docker run -i -t -p 35624:35624 -p 35625:35625 -p 35626:35626 -p 35627:35627 -p 35628:35628 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=35626 --rest-port=35624 --tcp-port=35625 --discv5-udp-port=35627 --rest-address=0.0.0.0 --nat=extip:172.18.144.233 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=4bed9bd276eafb9ef4cb96add24f1fd7e5efc1ed4fdc70d4b99d2b298fc96d9e --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=35628 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QGLmC8XCOvhm9Pjvtufl_7BF2fy3dUu4rvCkuYBSI1FJEft9Ox9tYB7z1SQ9dcMgW3gSS40nclSReUGbw7OK3C8CgmlkgnY0gmlwhKwSEsmKbXVsdGlhZGRyc5YACASsEhLJBpm1AAoErBISyQaZtt0DgnJzhQADAQAAiXNlY3AyNTZrMaED7Rr6ZI12EJeArhSmBsfvcHlXvCG7vcLPaI93DTJ8ZCCDdGNwgpm1g3VkcIKZt4V3YWt1MgE
2025-12-22 04:28:56.021 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.144.233 waku 19522e0675e8caae32e0ca39d09a9a77042f20c1cbdc93007b74e73e7b8a69b2
2025-12-22 04:28:56.053 DEBUG [src.node.docker_mananger] Container started with ID 19522e0675e8. Setting up logs at ./log/docker/node5_2025-12-22_04-28-50__ff6c0b6f-e075-499b-856f-78e96eb1591f__wakuorg_nwaku:latest.log
2025-12-22 04:28:56.053 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 35624
2025-12-22 04:28:56.054 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-22 04:28:57.055 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35624/health" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.064 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-22 04:28:57.066 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-22 04:28:57.067 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35624/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.072 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.144.233/tcp/35625/p2p/16Uiu2HAmL3i5f9LM8u7RSYm4xpUd7ZsCqwQmTcLdMsVBWJfju58C","/ip4/172.18.144.233/tcp/35626/ws/p2p/16Uiu2HAmL3i5f9LM8u7RSYm4xpUd7ZsCqwQmTcLdMsVBWJfju58C"],"enrUri":"enr:-L24QHtcn7NyA-xMpsL7yB89ZRDHn_hf5jVyuOI1cT4XLMeGPNX-HP9c4NX8G-saJfHEbd-lghxyoOhfd1jCekG-3DQCgmlkgnY0gmlwhKwSkOmKbXVsdGlhZGRyc5YACASsEpDpBospAAoErBKQ6QaLKt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDbdPFZv_3vSEyL8qmX2euCiER6IJFMFFDiBnO5i285XGDdGNwgospg3VkcIKLK4V3YWt1MgE"}'
2025-12-22 04:28:57.072 INFO [src.node.waku_node] REST service is ready !!
2025-12-22 04:28:57.073 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35624/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.18.201/tcp/39349/p2p/16Uiu2HAmUcYqMivU6175rMmmrTKojTzHjpB9dLYcKeyKL73NRzDM"]'
2025-12-22 04:28:57.076 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:57.076 DEBUG [src.steps.relay] Running fixture setup: subscribe_main_relay_nodes
2025-12-22 04:28:57.076 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39348/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2025-12-22 04:28:57.084 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:57.084 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:59673/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2025-12-22 04:28:57.089 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:57.089 DEBUG [src.steps.relay] Running fixture setup: subscribe_optional_relay_nodes
2025-12-22 04:28:57.090 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:31130/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2025-12-22 04:28:57.096 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:57.096 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:62432/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2025-12-22 04:28:57.104 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:57.104 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35624/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2025-12-22 04:28:57.110 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:57.111 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39348/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2025-12-22 04:28:57.121 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:57.124 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2025-12-22 04:28:57.224 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2025-12-22 04:28:57.225 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39348/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.228 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1766377737111092480,"ephemeral":false,"proof":""}]'
2025-12-22 04:28:57.229 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2025-12-22 04:28:57.229 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59673/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.232 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1766377737111092480,"ephemeral":false,"proof":""}]'
2025-12-22 04:28:57.233 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2025-12-22 04:28:57.233 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:31130/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.236 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1766377737111092480,"ephemeral":false,"proof":""}]'
2025-12-22 04:28:57.237 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2025-12-22 04:28:57.237 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:62432/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.239 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1766377737111092480,"ephemeral":false,"proof":""}]'
2025-12-22 04:28:57.241 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2025-12-22 04:28:57.241 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35624/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.243 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1766377737111092480,"ephemeral":false,"proof":""}]'
2025-12-22 04:28:57.244 INFO [src.steps.relay] WARM UP successful!!
2025-12-22 04:28:57.246 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39348/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2025-12-22 04:28:57.252 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:57.252 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2025-12-22 04:28:57.354 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2025-12-22 04:28:57.354 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39348/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.357 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1766377737246014622,"ephemeral":false,"proof":""}]'
2025-12-22 04:28:57.358 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2025-12-22 04:28:57.359 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59673/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.361 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1766377737246014622,"ephemeral":false,"proof":""}]'
2025-12-22 04:28:57.362 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2025-12-22 04:28:57.363 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:31130/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.365 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1766377737246014622,"ephemeral":false,"proof":""}]'
2025-12-22 04:28:57.367 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2025-12-22 04:28:57.367 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:62432/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.369 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1766377737246014622,"ephemeral":false,"proof":""}]'
2025-12-22 04:28:57.371 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2025-12-22 04:28:57.371 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35624/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.373 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1766377737246014622,"ephemeral":false,"proof":""}]'
2025-12-22 04:28:57.375 DEBUG [src.node.waku_node] Pausing container with id cde77687c71c
2025-12-22 04:28:57.385 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39348/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2025-12-22 04:28:57.391 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:57.391 DEBUG [src.node.waku_node] Unpause container with id cde77687c71c
2025-12-22 04:28:57.405 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39348/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2025-12-22 04:28:57.415 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-22 04:28:57.416 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59673/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-22 04:28:57.419 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1766377737375426123,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1766377737375503937,"ephemeral":false,"proof":""}]'
2025-12-22 04:28:57.422 DEBUG [tests.conftest] Running fixture teardown: test_setup
2025-12-22 04:28:57.423 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2025-12-22 04:28:57.423 DEBUG [src.node.waku_node] Stopping container with id 7705bfdbf5a8
2025-12-22 04:28:58.040 DEBUG [src.node.waku_node] Container stopped.
2025-12-22 04:28:58.043 DEBUG [src.node.waku_node] Stopping container with id cde77687c71c
2025-12-22 04:28:58.627 DEBUG [src.node.waku_node] Container stopped.
2025-12-22 04:28:58.627 DEBUG [src.node.waku_node] Stopping container with id a390dbe17018
2025-12-22 04:28:58.855 ERROR [src.node.docker_mananger] Max retries reached for container 7705bfdbf5a8. Exiting log stream.
2025-12-22 04:28:59.193 DEBUG [src.node.waku_node] Container stopped.
2025-12-22 04:28:59.193 DEBUG [src.node.waku_node] Stopping container with id 68f8cc864c66
2025-12-22 04:28:59.442 ERROR [src.node.docker_mananger] Max retries reached for container cde77687c71c. Exiting log stream.
2025-12-22 04:28:59.763 DEBUG [src.node.waku_node] Container stopped.
2025-12-22 04:28:59.763 DEBUG [src.node.waku_node] Stopping container with id 19522e0675e8
2025-12-22 04:29:00.008 ERROR [src.node.docker_mananger] Max retries reached for container a390dbe17018. Exiting log stream.
2025-12-22 04:29:00.290 DEBUG [src.node.waku_node] Container stopped.
2025-12-22 04:29:00.291 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2025-12-22 04:29:00.308 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-22 04:29:00.315 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-22 04:29:00.321 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-22 04:29:00.327 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-22 04:29:00.332 DEBUG [src.node.docker_mananger] No errors found in the waku logs.