159 lines
32 KiB
Plaintext

2026-03-11 04:33:15.541 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-11 04:33:15.542 DEBUG [tests.conftest] Running test: test_relay_4_nodes_two_publishers_compare_latency with id: 2026-03-11_04-33-15__9ab7b237-6a03-4344-8c92-1f024c7b5bfc
2026-03-11 04:33:15.542 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-11 04:33:15.542 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-11 04:33:15.549 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:33:15.549 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-11_04-33-15__9ab7b237-6a03-4344-8c92-1f024c7b5bfc__wakuorg_nwaku:latest.log
2026-03-11 04:33:15.555 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:33:15.555 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-11_04-33-15__9ab7b237-6a03-4344-8c92-1f024c7b5bfc__wakuorg_nwaku:latest.log
2026-03-11 04:33:15.562 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:33:15.562 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-11_04-33-15__9ab7b237-6a03-4344-8c92-1f024c7b5bfc__wakuorg_nwaku:latest.log
2026-03-11 04:33:15.569 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:33:15.569 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-11_04-33-15__9ab7b237-6a03-4344-8c92-1f024c7b5bfc__wakuorg_nwaku:latest.log
2026-03-11 04:33:15.569 INFO [tests.e2e.test_network_conditions] Starting 4 nodes
2026-03-11 04:33:15.569 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:33:15.569 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:33:15.571 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:33:15.571 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.139.21
2026-03-11 04:33:15.571 DEBUG [src.node.docker_mananger] Generated ports ['48865', '48866', '48867', '48868', '48869']
2026-03-11 04:33:15.571 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:33:15.571 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:33:15.571 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:33:15.571 DEBUG [src.node.docker_mananger] docker run -i -t -p 48865:48865 -p 48866:48866 -p 48867:48867 -p 48868:48868 -p 48869:48869 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=48867 --rest-port=48865 --tcp-port=48866 --discv5-udp-port=48868 --rest-address=0.0.0.0 --nat=extip:172.18.139.21 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=b59aed4aceae8c03cb83bfa3daaaf3ebef00799d1dfcbecaa6eaad20a1bde959 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=48869 --metrics-logging=true --relay=true
2026-03-11 04:33:15.740 ERROR [src.node.docker_mananger] Max retries reached for container f79a764e127a. Exiting log stream.
2026-03-11 04:33:15.759 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.139.21 waku 8fbd521ed2527b594deae38cc9b23a10e5216b0cf1f134513e2b03cff158df96
2026-03-11 04:33:15.796 DEBUG [src.node.docker_mananger] Container started with ID 8fbd521ed252. Setting up logs at ./log/docker/node1_2026-03-11_04-33-15__9ab7b237-6a03-4344-8c92-1f024c7b5bfc__wakuorg_nwaku:latest.log
2026-03-11 04:33:15.796 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 48865
2026-03-11 04:33:15.797 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:33:16.321 ERROR [src.node.docker_mananger] Max retries reached for container 2790c46e3a63. Exiting log stream.
2026-03-11 04:33:16.798 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:48865/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:33:16.801 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:33:16.801 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:33:16.801 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:48865/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:33:16.803 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.139.21/tcp/48866/p2p/16Uiu2HAmJQHZ7XhveHKiJyWT2u4Qd84CzpkGm4e9HgcpNGXxmzo1","/ip4/172.18.139.21/tcp/48867/ws/p2p/16Uiu2HAmJQHZ7XhveHKiJyWT2u4Qd84CzpkGm4e9HgcpNGXxmzo1"],"enrUri":"enr:-L24QJI-WvqYeza3RzXqNGAPWIHMNy9PanMGl9HbtPBLEVRyGCj4H9Kq_XN_on6HzWTwb3vg8WraZWj6n3oHCgMJfiQCgmlkgnY0gmlwhKwSixWKbXVsdGlhZGRyc5YACASsEosVBr7iAAoErBKLFQa-490DgnJzhQADAQAAiXNlY3AyNTZrMaEDVWHcpEKxWlOX0GQudXem6zd5jbsjY0WLZZ3aXAKAwMCDdGNwgr7ig3VkcIK-5IV3YWt1MgE"}'
2026-03-11 04:33:16.804 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:33:16.804 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:33:16.804 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:33:16.805 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:33:16.806 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.12.159
2026-03-11 04:33:16.806 DEBUG [src.node.docker_mananger] Generated ports ['55485', '55486', '55487', '55488', '55489']
2026-03-11 04:33:16.806 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:33:16.806 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:33:16.806 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:33:16.806 DEBUG [src.node.docker_mananger] docker run -i -t -p 55485:55485 -p 55486:55486 -p 55487:55487 -p 55488:55488 -p 55489:55489 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=55487 --rest-port=55485 --tcp-port=55486 --discv5-udp-port=55488 --rest-address=0.0.0.0 --nat=extip:172.18.12.159 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=fdbd1cf7a975eda6eb17f1ba4ab132c91dacf51e192a4dc202eaefefac9aec5b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=55489 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QJI-WvqYeza3RzXqNGAPWIHMNy9PanMGl9HbtPBLEVRyGCj4H9Kq_XN_on6HzWTwb3vg8WraZWj6n3oHCgMJfiQCgmlkgnY0gmlwhKwSixWKbXVsdGlhZGRyc5YACASsEosVBr7iAAoErBKLFQa-490DgnJzhQADAQAAiXNlY3AyNTZrMaEDVWHcpEKxWlOX0GQudXem6zd5jbsjY0WLZZ3aXAKAwMCDdGNwgr7ig3VkcIK-5IV3YWt1MgE
2026-03-11 04:33:16.991 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.12.159 waku 2891c45ccf31d92dc7ac98b3ae97873911b6d86a0bd25b896275d8f0ced3fc1c
2026-03-11 04:33:17.026 DEBUG [src.node.docker_mananger] Container started with ID 2891c45ccf31. Setting up logs at ./log/docker/node2_2026-03-11_04-33-15__9ab7b237-6a03-4344-8c92-1f024c7b5bfc__wakuorg_nwaku:latest.log
2026-03-11 04:33:17.026 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 55485
2026-03-11 04:33:17.026 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:33:18.027 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:55485/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:33:18.050 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:33:18.052 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:33:18.055 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:55485/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:33:18.061 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.12.159/tcp/55486/p2p/16Uiu2HAm1fhCaXhEVeyt7KRSP2C5FWPu6kq7WYUuJBbFeskpeCSY","/ip4/172.18.12.159/tcp/55487/ws/p2p/16Uiu2HAm1fhCaXhEVeyt7KRSP2C5FWPu6kq7WYUuJBbFeskpeCSY"],"enrUri":"enr:-L24QNB4X1MBqv1Msd8VA9H5DxON3b8KA1izpjv1emKNnotdTwW2vyU5flJRLeL2i_zNqC5evnJoDwCkS3So25YIJ_cCgmlkgnY0gmlwhKwSDJ-KbXVsdGlhZGRyc5YACASsEgyfBti-AAoErBIMnwbYv90DgnJzhQADAQAAiXNlY3AyNTZrMaECXL2iAbSIAURUDXrUCYUwXMBa_hHN2VKhryLPFz3PFO2DdGNwgti-g3VkcILYwIV3YWt1MgE"}'
2026-03-11 04:33:18.061 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:33:18.061 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:33:18.062 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:33:18.065 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:33:18.065 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.232.84
2026-03-11 04:33:18.065 DEBUG [src.node.docker_mananger] Generated ports ['20263', '20264', '20265', '20266', '20267']
2026-03-11 04:33:18.065 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:33:18.065 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:33:18.065 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:33:18.066 DEBUG [src.node.docker_mananger] docker run -i -t -p 20263:20263 -p 20264:20264 -p 20265:20265 -p 20266:20266 -p 20267:20267 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=20265 --rest-port=20263 --tcp-port=20264 --discv5-udp-port=20266 --rest-address=0.0.0.0 --nat=extip:172.18.232.84 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=a0e62af4218f7bf5ed64dc00b6c82c5edf90ea16ea0cfb578e93f508e4e2ddfe --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=20267 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNB4X1MBqv1Msd8VA9H5DxON3b8KA1izpjv1emKNnotdTwW2vyU5flJRLeL2i_zNqC5evnJoDwCkS3So25YIJ_cCgmlkgnY0gmlwhKwSDJ-KbXVsdGlhZGRyc5YACASsEgyfBti-AAoErBIMnwbYv90DgnJzhQADAQAAiXNlY3AyNTZrMaECXL2iAbSIAURUDXrUCYUwXMBa_hHN2VKhryLPFz3PFO2DdGNwgti-g3VkcILYwIV3YWt1MgE
2026-03-11 04:33:18.262 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.232.84 waku b21ff2c0042819fafd26a61fa05b256ecdaf1f9b6a089163b100f1628ed6e599
2026-03-11 04:33:18.299 DEBUG [src.node.docker_mananger] Container started with ID b21ff2c00428. Setting up logs at ./log/docker/node3_2026-03-11_04-33-15__9ab7b237-6a03-4344-8c92-1f024c7b5bfc__wakuorg_nwaku:latest.log
2026-03-11 04:33:18.300 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 20263
2026-03-11 04:33:18.300 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:33:19.301 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:20263/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:33:19.320 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:33:19.321 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:33:19.321 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:20263/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:33:19.327 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.232.84/tcp/20264/p2p/16Uiu2HAmHJVAnNDpbG7k7uDEsGR1Aj2KhDk31xDSNCS71zF4r6P1","/ip4/172.18.232.84/tcp/20265/ws/p2p/16Uiu2HAmHJVAnNDpbG7k7uDEsGR1Aj2KhDk31xDSNCS71zF4r6P1"],"enrUri":"enr:-L24QM7CPT6BfcInTllNtDaH8i1SUUTZDSquP7g2hZz14ypWFCE3efasC8ZztVN2GYjGGbOCbBw7FAWHFjFztK4WzGICgmlkgnY0gmlwhKwS6FSKbXVsdGlhZGRyc5YACASsEuhUBk8oAAoErBLoVAZPKd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDRQnSvjkhwFkAUuIORxulGJw1U2O5vz-A0Gb9QwExdoiDdGNwgk8og3VkcIJPKoV3YWt1MgE"}'
2026-03-11 04:33:19.328 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:33:19.328 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:33:19.328 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:33:19.329 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:33:19.330 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.23.13
2026-03-11 04:33:19.330 DEBUG [src.node.docker_mananger] Generated ports ['10060', '10061', '10062', '10063', '10064']
2026-03-11 04:33:19.330 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:33:19.330 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:33:19.330 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:33:19.330 DEBUG [src.node.docker_mananger] docker run -i -t -p 10060:10060 -p 10061:10061 -p 10062:10062 -p 10063:10063 -p 10064:10064 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=10062 --rest-port=10060 --tcp-port=10061 --discv5-udp-port=10063 --rest-address=0.0.0.0 --nat=extip:172.18.23.13 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=8fc91e21f201efb40ecbd03ba72058b76e6db9502bf73f24aaa3f82cbfbe0789 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=10064 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QM7CPT6BfcInTllNtDaH8i1SUUTZDSquP7g2hZz14ypWFCE3efasC8ZztVN2GYjGGbOCbBw7FAWHFjFztK4WzGICgmlkgnY0gmlwhKwS6FSKbXVsdGlhZGRyc5YACASsEuhUBk8oAAoErBLoVAZPKd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDRQnSvjkhwFkAUuIORxulGJw1U2O5vz-A0Gb9QwExdoiDdGNwgk8og3VkcIJPKoV3YWt1MgE
2026-03-11 04:33:19.533 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.23.13 waku 28738cb21e3b9919ccc307d0468709b771a1326a8dc60de69e7dbd21ba669128
2026-03-11 04:33:19.568 DEBUG [src.node.docker_mananger] Container started with ID 28738cb21e3b. Setting up logs at ./log/docker/node4_2026-03-11_04-33-15__9ab7b237-6a03-4344-8c92-1f024c7b5bfc__wakuorg_nwaku:latest.log
2026-03-11 04:33:19.568 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 10060
2026-03-11 04:33:19.569 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:33:20.570 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10060/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:33:20.586 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:33:20.587 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:33:20.587 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10060/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:33:20.596 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.23.13/tcp/10061/p2p/16Uiu2HAmGraLkxhABY49A92g1MLg4u7T5gKrJhXDRHnkaw2WxsAs","/ip4/172.18.23.13/tcp/10062/ws/p2p/16Uiu2HAmGraLkxhABY49A92g1MLg4u7T5gKrJhXDRHnkaw2WxsAs"],"enrUri":"enr:-L24QNTXREuI3YJmLTpYLa3qe0cgvBENvGYtdfpWXUi2Ji5mX6SHO_mbSyc4J7VAPl4wVKyvZ66AAf1Nu2_CAXcDbO0CgmlkgnY0gmlwhKwSFw2KbXVsdGlhZGRyc5YACASsEhcNBidNAAoErBIXDQYnTt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDPmaR9uU2xsA7W9YWyDyiLe2kjOvYvv3A4gP03c-eq0yDdGNwgidNg3VkcIInT4V3YWt1MgE"}'
2026-03-11 04:33:20.596 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:33:20.597 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48865/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-11 04:33:20.602 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:33:20.602 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:55485/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-11 04:33:20.610 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:33:20.610 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:20263/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-11 04:33:20.617 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:33:20.618 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:10060/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-11 04:33:20.628 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:33:20.628 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:48865/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:33:20.632 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.12.159/tcp/52192/p2p/16Uiu2HAm1fhCaXhEVeyt7KRSP2C5FWPu6kq7WYUuJBbFeskpeCSY","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"UnknownOrigin"}]'
2026-03-11 04:33:20.633 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:55485/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:33:20.636 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.139.21/tcp/48866/p2p/16Uiu2HAmJQHZ7XhveHKiJyWT2u4Qd84CzpkGm4e9HgcpNGXxmzo1","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"Discv5"},{"multiaddr":"/ip4/172.18.232.84/tcp/60012/p2p/16Uiu2HAmHJVAnNDpbG7k7uDEsGR1Aj2KhDk31xDSNCS71zF4r6P1","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"UnknownOrigin"}]'
2026-03-11 04:33:20.636 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:20263/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:33:20.639 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.12.159/tcp/55486/p2p/16Uiu2HAm1fhCaXhEVeyt7KRSP2C5FWPu6kq7WYUuJBbFeskpeCSY","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"Discv5"},{"multiaddr":"/ip4/172.18.23.13/tcp/46028/p2p/16Uiu2HAmGraLkxhABY49A92g1MLg4u7T5gKrJhXDRHnkaw2WxsAs","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"UnknownOrigin"}]'
2026-03-11 04:33:20.639 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10060/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:33:20.641 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.232.84/tcp/20264/p2p/16Uiu2HAmHJVAnNDpbG7k7uDEsGR1Aj2KhDk31xDSNCS71zF4r6P1","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"Discv5"}]'
2026-03-11 04:33:20.641 DEBUG [src.libs.common] Sleeping for 60 seconds
2026-03-11 04:34:20.642 DEBUG [tests.e2e.test_network_conditions] Applying 3000ms latency on node1 only
2026-03-11 04:34:20.644 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '4292', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-11 04:34:20.697 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '4292', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'delay', '3000ms']
2026-03-11 04:34:20.710 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10060/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:34:20.713 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[]'
2026-03-11 04:34:20.713 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48865/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:34:26.722 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:34:26.723 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:55485/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:34:26.730 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:34:26.730 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-11 04:34:26.934 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48865/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:34:32.942 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:34:32.942 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:55485/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:34:32.954 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:34:32.957 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-11 04:34:33.161 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48865/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:34:39.170 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:34:39.170 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:55485/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:34:39.181 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:34:39.181 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-11 04:34:39.381 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48865/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:34:45.389 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:34:45.389 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:55485/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:34:45.398 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:34:45.401 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-11 04:34:45.603 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48865/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:34:51.610 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:34:51.611 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:55485/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:34:51.620 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:34:51.621 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-11 04:34:51.823 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10060/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:34:51.826 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773203660713566331,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773203666723165309,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773203666934512685,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773203672942498910,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773203673161645979,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773203679170745536,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773203679381687682,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773203685389895706,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773203685603668024,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773203691611247543,"ephemeral":false,"proof":""}]'
2026-03-11 04:34:51.828 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '4292', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-11 04:34:51.839 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-11 04:34:51.839 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-11 04:34:51.840 DEBUG [src.node.waku_node] Stopping container with id 8fbd521ed252
2026-03-11 04:34:52.418 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:34:52.420 DEBUG [src.node.waku_node] Stopping container with id 2891c45ccf31
2026-03-11 04:34:52.973 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:34:52.976 DEBUG [src.node.waku_node] Stopping container with id b21ff2c00428
2026-03-11 04:34:53.471 ERROR [src.node.docker_mananger] Max retries reached for container 8fbd521ed252. Exiting log stream.
2026-03-11 04:34:53.532 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:34:53.535 DEBUG [src.node.waku_node] Stopping container with id 28738cb21e3b
2026-03-11 04:34:53.787 ERROR [src.node.docker_mananger] Max retries reached for container 2891c45ccf31. Exiting log stream.
2026-03-11 04:34:54.082 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:34:54.084 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-11 04:34:54.221 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:34:54.295 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:34:54.366 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:34:54.441 DEBUG [src.node.docker_mananger] No errors found in the waku logs.