168 lines
34 KiB
Plaintext

2026-03-08 04:34:48.053 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-08 04:34:48.053 DEBUG [tests.conftest] Running test: test_relay_packet_loss_correlated_vs_uncorrelated with id: 2026-03-08_04-34-48__f1afa5f0-246f-42bb-a8d5-4fb58b522059
2026-03-08 04:34:48.053 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-08 04:34:48.054 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-08 04:34:48.061 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-08 04:34:48.061 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-08_04-34-48__f1afa5f0-246f-42bb-a8d5-4fb58b522059__wakuorg_nwaku:latest.log
2026-03-08 04:34:48.068 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-08 04:34:48.068 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-08_04-34-48__f1afa5f0-246f-42bb-a8d5-4fb58b522059__wakuorg_nwaku:latest.log
2026-03-08 04:34:48.076 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-08 04:34:48.076 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-08_04-34-48__f1afa5f0-246f-42bb-a8d5-4fb58b522059__wakuorg_nwaku:latest.log
2026-03-08 04:34:48.083 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-08 04:34:48.083 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-08_04-34-48__f1afa5f0-246f-42bb-a8d5-4fb58b522059__wakuorg_nwaku:latest.log
2026-03-08 04:34:48.083 DEBUG [src.node.waku_node] Starting Node...
2026-03-08 04:34:48.083 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-08 04:34:48.085 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-08 04:34:48.085 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.69.138
2026-03-08 04:34:48.085 DEBUG [src.node.docker_mananger] Generated ports ['57214', '57215', '57216', '57217', '57218']
2026-03-08 04:34:48.085 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-08 04:34:48.085 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-08 04:34:48.086 DEBUG [src.node.waku_node] Using volumes []
2026-03-08 04:34:48.086 DEBUG [src.node.docker_mananger] docker run -i -t -p 57214:57214 -p 57215:57215 -p 57216:57216 -p 57217:57217 -p 57218:57218 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=57216 --rest-port=57214 --tcp-port=57215 --discv5-udp-port=57217 --rest-address=0.0.0.0 --nat=extip:172.18.69.138 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=8e49f2f97e241db66fdacd23720e61ef752baa83a6af9ea82dce56d6d02eddbb --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=57218 --metrics-logging=true --relay=true
2026-03-08 04:34:48.253 ERROR [src.node.docker_mananger] Max retries reached for container 4e229645d12e. Exiting log stream.
2026-03-08 04:34:48.285 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.69.138 waku d159e6856af2439520fafb9492f47d31307ed690e20f2efe62f7dfec26e1f40b
2026-03-08 04:34:48.320 DEBUG [src.node.docker_mananger] Container started with ID d159e6856af2. Setting up logs at ./log/docker/node1_2026-03-08_04-34-48__f1afa5f0-246f-42bb-a8d5-4fb58b522059__wakuorg_nwaku:latest.log
2026-03-08 04:34:48.320 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 57214
2026-03-08 04:34:48.320 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-08 04:34:48.845 ERROR [src.node.docker_mananger] Max retries reached for container fbbb028454cb. Exiting log stream.
2026-03-08 04:34:49.321 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57214/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:49.324 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-08 04:34:49.324 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-08 04:34:49.324 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57214/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:49.327 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.69.138/tcp/57215/p2p/16Uiu2HAkvMGRt2uNg11hMMhGxW9FVtjeC6XRKpryJCFAQx4MrZMs","/ip4/172.18.69.138/tcp/57216/ws/p2p/16Uiu2HAkvMGRt2uNg11hMMhGxW9FVtjeC6XRKpryJCFAQx4MrZMs"],"enrUri":"enr:-L24QLdi15NgNwO8Gt3CKQmG8NobC0igvgbUPOICh_RORFXCLV9mzDt3cDawxYFi1sCC0WGefPNCNeuCQaOEl1rXXhgCgmlkgnY0gmlwhKwSRYqKbXVsdGlhZGRyc5YACASsEkWKBt9_AAoErBJFigbfgN0DgnJzhQADAQAAiXNlY3AyNTZrMaECDbq8QTfKDKq0RIg2uymZGaFTyzIWyDxmHIyUjY42I0KDdGNwgt9_g3VkcILfgYV3YWt1MgE"}'
2026-03-08 04:34:49.327 INFO [src.node.waku_node] REST service is ready !!
2026-03-08 04:34:49.327 DEBUG [src.node.waku_node] Starting Node...
2026-03-08 04:34:49.327 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-08 04:34:49.329 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-08 04:34:49.329 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.20.63
2026-03-08 04:34:49.329 DEBUG [src.node.docker_mananger] Generated ports ['19596', '19597', '19598', '19599', '19600']
2026-03-08 04:34:49.329 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-08 04:34:49.329 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-08 04:34:49.329 DEBUG [src.node.waku_node] Using volumes []
2026-03-08 04:34:49.330 DEBUG [src.node.docker_mananger] docker run -i -t -p 19596:19596 -p 19597:19597 -p 19598:19598 -p 19599:19599 -p 19600:19600 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=19598 --rest-port=19596 --tcp-port=19597 --discv5-udp-port=19599 --rest-address=0.0.0.0 --nat=extip:172.18.20.63 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9e569225ba7da6bbea74ccebbf8b660a1124fb657c0d43f8b2acd744e0b3d87d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=19600 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QLdi15NgNwO8Gt3CKQmG8NobC0igvgbUPOICh_RORFXCLV9mzDt3cDawxYFi1sCC0WGefPNCNeuCQaOEl1rXXhgCgmlkgnY0gmlwhKwSRYqKbXVsdGlhZGRyc5YACASsEkWKBt9_AAoErBJFigbfgN0DgnJzhQADAQAAiXNlY3AyNTZrMaECDbq8QTfKDKq0RIg2uymZGaFTyzIWyDxmHIyUjY42I0KDdGNwgt9_g3VkcILfgYV3YWt1MgE
2026-03-08 04:34:49.533 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.20.63 waku f46a56fe0c0fe85e4eda8de0acc27753702874cdb2488c109b7eee70acedbd10
2026-03-08 04:34:49.568 DEBUG [src.node.docker_mananger] Container started with ID f46a56fe0c0f. Setting up logs at ./log/docker/node2_2026-03-08_04-34-48__f1afa5f0-246f-42bb-a8d5-4fb58b522059__wakuorg_nwaku:latest.log
2026-03-08 04:34:49.569 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 19596
2026-03-08 04:34:49.569 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-08 04:34:50.569 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19596/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:50.594 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-08 04:34:50.595 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-08 04:34:50.595 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19596/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:50.603 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.20.63/tcp/19597/p2p/16Uiu2HAmT5AuJhQZqbS9WgosnA8n25esndTgF7MNrQJSW9t2tFuT","/ip4/172.18.20.63/tcp/19598/ws/p2p/16Uiu2HAmT5AuJhQZqbS9WgosnA8n25esndTgF7MNrQJSW9t2tFuT"],"enrUri":"enr:-L24QLSH5IWdiCS2Q69OCbLj36GANxdur8121Ud_X1zAa43fWYRbkeQ8WZvHA2_CoA38CF-m6UlCeA8PEhcC3TCDVlACgmlkgnY0gmlwhKwSFD-KbXVsdGlhZGRyc5YACASsEhQ_BkyNAAoErBIUPwZMjt0DgnJzhQADAQAAiXNlY3AyNTZrMaED1jV80Yjpjr7PG6tpQaEHRDpT9DPyASxzli8c2-ZJ_0KDdGNwgkyNg3VkcIJMj4V3YWt1MgE"}'
2026-03-08 04:34:50.603 INFO [src.node.waku_node] REST service is ready !!
2026-03-08 04:34:50.604 DEBUG [src.node.waku_node] Starting Node...
2026-03-08 04:34:50.604 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-08 04:34:50.607 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-08 04:34:50.608 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.48.31
2026-03-08 04:34:50.608 DEBUG [src.node.docker_mananger] Generated ports ['58269', '58270', '58271', '58272', '58273']
2026-03-08 04:34:50.608 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-08 04:34:50.608 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-08 04:34:50.609 DEBUG [src.node.waku_node] Using volumes []
2026-03-08 04:34:50.609 DEBUG [src.node.docker_mananger] docker run -i -t -p 58269:58269 -p 58270:58270 -p 58271:58271 -p 58272:58272 -p 58273:58273 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=58271 --rest-port=58269 --tcp-port=58270 --discv5-udp-port=58272 --rest-address=0.0.0.0 --nat=extip:172.18.48.31 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=38fddfca04d7872b8c05e47daff6a18bef65f1ffdab9af5d9f47adabc8eb1ba2 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=58273 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QLSH5IWdiCS2Q69OCbLj36GANxdur8121Ud_X1zAa43fWYRbkeQ8WZvHA2_CoA38CF-m6UlCeA8PEhcC3TCDVlACgmlkgnY0gmlwhKwSFD-KbXVsdGlhZGRyc5YACASsEhQ_BkyNAAoErBIUPwZMjt0DgnJzhQADAQAAiXNlY3AyNTZrMaED1jV80Yjpjr7PG6tpQaEHRDpT9DPyASxzli8c2-ZJ_0KDdGNwgkyNg3VkcIJMj4V3YWt1MgE
2026-03-08 04:34:50.810 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.48.31 waku d865f17669652eb1bf4d4df3e971a76e16fa63cc28ca07fa25154d31342406d5
2026-03-08 04:34:50.844 DEBUG [src.node.docker_mananger] Container started with ID d865f1766965. Setting up logs at ./log/docker/node3_2026-03-08_04-34-48__f1afa5f0-246f-42bb-a8d5-4fb58b522059__wakuorg_nwaku:latest.log
2026-03-08 04:34:50.844 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 58269
2026-03-08 04:34:50.844 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-08 04:34:51.845 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58269/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:51.863 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-08 04:34:51.866 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-08 04:34:51.867 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58269/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:51.873 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.48.31/tcp/58270/p2p/16Uiu2HAm2SyM7HRv8Y7n1JbEHczNj1ZsenXQRe9CeBXyVDgGqbB8","/ip4/172.18.48.31/tcp/58271/ws/p2p/16Uiu2HAm2SyM7HRv8Y7n1JbEHczNj1ZsenXQRe9CeBXyVDgGqbB8"],"enrUri":"enr:-L24QLk3gjrpviOI_RnRWYMKAGv9-jVK_qQwAJ9txL8SLBx-QFLqloe-OVzffnqr3BTS5o0N05J3X0gB-vyS1MrZWH4CgmlkgnY0gmlwhKwSMB-KbXVsdGlhZGRyc5YACASsEjAfBuOeAAoErBIwHwbjn90DgnJzhQADAQAAiXNlY3AyNTZrMaECaFcG3GlMTCEWy-ADnnz19gkFD4GkAPK-jtxZv06sDOODdGNwguOeg3VkcILjoIV3YWt1MgE"}'
2026-03-08 04:34:51.874 INFO [src.node.waku_node] REST service is ready !!
2026-03-08 04:34:51.874 DEBUG [src.node.waku_node] Starting Node...
2026-03-08 04:34:51.875 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-08 04:34:51.876 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-08 04:34:51.876 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.67.45
2026-03-08 04:34:51.877 DEBUG [src.node.docker_mananger] Generated ports ['39306', '39307', '39308', '39309', '39310']
2026-03-08 04:34:51.877 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-08 04:34:51.877 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-08 04:34:51.877 DEBUG [src.node.waku_node] Using volumes []
2026-03-08 04:34:51.877 DEBUG [src.node.docker_mananger] docker run -i -t -p 39306:39306 -p 39307:39307 -p 39308:39308 -p 39309:39309 -p 39310:39310 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=39308 --rest-port=39306 --tcp-port=39307 --discv5-udp-port=39309 --rest-address=0.0.0.0 --nat=extip:172.18.67.45 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=ccfbfbaa871d4a3d17a83a7ef6a0f448e8becf471e933a2ddafc4faab6c40a3c --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=39310 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QLk3gjrpviOI_RnRWYMKAGv9-jVK_qQwAJ9txL8SLBx-QFLqloe-OVzffnqr3BTS5o0N05J3X0gB-vyS1MrZWH4CgmlkgnY0gmlwhKwSMB-KbXVsdGlhZGRyc5YACASsEjAfBuOeAAoErBIwHwbjn90DgnJzhQADAQAAiXNlY3AyNTZrMaECaFcG3GlMTCEWy-ADnnz19gkFD4GkAPK-jtxZv06sDOODdGNwguOeg3VkcILjoIV3YWt1MgE
2026-03-08 04:34:52.076 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.67.45 waku ab197632d9d1d1db671540cb411df1a6327b0a5fc61c79b32bab2c412259b62c
2026-03-08 04:34:52.112 DEBUG [src.node.docker_mananger] Container started with ID ab197632d9d1. Setting up logs at ./log/docker/node4_2026-03-08_04-34-48__f1afa5f0-246f-42bb-a8d5-4fb58b522059__wakuorg_nwaku:latest.log
2026-03-08 04:34:52.112 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 39306
2026-03-08 04:34:52.113 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-08 04:34:53.114 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39306/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:53.129 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-08 04:34:53.131 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-08 04:34:53.133 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39306/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:53.137 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.67.45/tcp/39307/p2p/16Uiu2HAm7EB8orYh3V4eQ3qQfvbmnbwGK9zWGurMbZtMSayjxoq1","/ip4/172.18.67.45/tcp/39308/ws/p2p/16Uiu2HAm7EB8orYh3V4eQ3qQfvbmnbwGK9zWGurMbZtMSayjxoq1"],"enrUri":"enr:-L24QGvLGriQOleTVzlSTf7Ly-RRDKWnclCMY7pBW1S4B87lCPGc-bX2GuY9kNJ1_gslnCtapUMA5wsKAWW1X4vkPzUCgmlkgnY0gmlwhKwSQy2KbXVsdGlhZGRyc5YACASsEkMtBpmLAAoErBJDLQaZjN0DgnJzhQADAQAAiXNlY3AyNTZrMaECr1o9U9w1VD7TQSFEMc_p8cqvmM3EaBIjPoos1M4s7FCDdGNwgpmLg3VkcIKZjYV3YWt1MgE"}'
2026-03-08 04:34:53.138 INFO [src.node.waku_node] REST service is ready !!
2026-03-08 04:34:53.138 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57214/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-08 04:34:53.145 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:53.146 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19596/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-08 04:34:53.153 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:53.155 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58269/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-08 04:34:53.162 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:53.163 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39306/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-08 04:34:53.170 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:53.171 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57214/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:53.175 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.20.63/tcp/39198/p2p/16Uiu2HAmT5AuJhQZqbS9WgosnA8n25esndTgF7MNrQJSW9t2tFuT","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"UnknownOrigin"}]'
2026-03-08 04:34:53.175 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19596/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:53.177 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.69.138/tcp/57215/p2p/16Uiu2HAkvMGRt2uNg11hMMhGxW9FVtjeC6XRKpryJCFAQx4MrZMs","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"Discv5"},{"multiaddr":"/ip4/172.18.48.31/tcp/38432/p2p/16Uiu2HAm2SyM7HRv8Y7n1JbEHczNj1ZsenXQRe9CeBXyVDgGqbB8","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"UnknownOrigin"}]'
2026-03-08 04:34:53.177 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58269/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:53.180 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.67.45/tcp/55856/p2p/16Uiu2HAm7EB8orYh3V4eQ3qQfvbmnbwGK9zWGurMbZtMSayjxoq1","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.20.63/tcp/19597/p2p/16Uiu2HAmT5AuJhQZqbS9WgosnA8n25esndTgF7MNrQJSW9t2tFuT","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"Discv5"}]'
2026-03-08 04:34:53.180 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39306/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:53.182 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.48.31/tcp/58270/p2p/16Uiu2HAm2SyM7HRv8Y7n1JbEHczNj1ZsenXQRe9CeBXyVDgGqbB8","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"Discv5"}]'
2026-03-08 04:34:53.182 DEBUG [src.libs.common] Sleeping for 20 seconds
2026-03-08 04:35:13.184 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7548', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-08 04:35:13.198 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7548', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'loss', '50.0%']
2026-03-08 04:35:13.210 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7548', '-n', 'tc', '-s', 'qdisc', 'show', 'dev', 'eth0']
2026-03-08 04:35:13.220 DEBUG [src.steps.network_conditions] tc -s qdisc show dev eth0:
qdisc netem 8002: root refcnt 5 limit 1000 loss 50%
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
2026-03-08 04:35:13.220 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39306/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:35:13.223 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[]'
2026-03-08 04:35:13.223 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57214/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:35:15.068 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:35:15.068 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57214/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:35:16.106 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:35:16.109 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57214/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:35:18.564 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:35:18.564 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57214/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:35:19.828 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:35:19.828 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57214/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:35:23.499 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:35:23.500 DEBUG [src.libs.common] Sleeping for 30.0 seconds
2026-03-08 04:35:53.500 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39306/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:35:53.503 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772944513223804912,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772944515068817037,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772944516109614840,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772944518564796620,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772944519828767358,"ephemeral":false,"proof":""}]'
2026-03-08 04:35:53.505 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7548', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-08 04:35:53.516 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7548', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-08 04:35:53.526 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7548', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'loss', '50.0%', '75.0%']
2026-03-08 04:35:53.535 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39306/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:35:53.538 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[]'
2026-03-08 04:35:53.538 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57214/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:35:55.012 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:35:55.012 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57214/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:36:00.137 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:36:00.141 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57214/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:36:02.186 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:36:02.187 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57214/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:36:03.214 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:36:03.216 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57214/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:36:03.228 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:36:03.231 DEBUG [src.libs.common] Sleeping for 30.0 seconds
2026-03-08 04:36:33.234 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39306/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:36:33.236 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772944553538323061,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772944555012527891,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772944560140895591,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772944562187451022,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772944563216863090,"ephemeral":false,"proof":""}]'
2026-03-08 04:36:33.238 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7548', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-08 04:36:33.248 DEBUG [tests.e2e.test_network_conditions] uncorrelated=5 correlated=5
2026-03-08 04:36:33.250 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-08 04:36:33.250 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-08 04:36:33.251 DEBUG [src.node.waku_node] Stopping container with id d159e6856af2
2026-03-08 04:36:33.832 DEBUG [src.node.waku_node] Container stopped.
2026-03-08 04:36:33.833 DEBUG [src.node.waku_node] Stopping container with id f46a56fe0c0f
2026-03-08 04:36:34.379 DEBUG [src.node.waku_node] Container stopped.
2026-03-08 04:36:34.380 DEBUG [src.node.waku_node] Stopping container with id d865f1766965
2026-03-08 04:36:34.645 ERROR [src.node.docker_mananger] Max retries reached for container d159e6856af2. Exiting log stream.
2026-03-08 04:36:34.939 DEBUG [src.node.waku_node] Container stopped.
2026-03-08 04:36:34.939 DEBUG [src.node.waku_node] Stopping container with id ab197632d9d1
2026-03-08 04:36:35.192 ERROR [src.node.docker_mananger] Max retries reached for container f46a56fe0c0f. Exiting log stream.
2026-03-08 04:36:35.493 DEBUG [src.node.waku_node] Container stopped.
2026-03-08 04:36:35.496 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-08 04:36:35.570 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-08 04:36:35.648 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-08 04:36:35.726 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-08 04:36:35.804 DEBUG [src.node.docker_mananger] No errors found in the waku logs.