168 lines
34 KiB
Plaintext

2026-03-12 04:35:56.266 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-12 04:35:56.266 DEBUG [tests.conftest] Running test: test_relay_packet_loss_correlated_vs_uncorrelated with id: 2026-03-12_04-35-56__6621863f-c62c-4624-91a7-258290dc259a
2026-03-12 04:35:56.267 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-12 04:35:56.267 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-12 04:35:56.273 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-12 04:35:56.274 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-12_04-35-56__6621863f-c62c-4624-91a7-258290dc259a__wakuorg_nwaku:latest.log
2026-03-12 04:35:56.280 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-12 04:35:56.280 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-12_04-35-56__6621863f-c62c-4624-91a7-258290dc259a__wakuorg_nwaku:latest.log
2026-03-12 04:35:56.286 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-12 04:35:56.286 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-12_04-35-56__6621863f-c62c-4624-91a7-258290dc259a__wakuorg_nwaku:latest.log
2026-03-12 04:35:56.292 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-12 04:35:56.292 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-12_04-35-56__6621863f-c62c-4624-91a7-258290dc259a__wakuorg_nwaku:latest.log
2026-03-12 04:35:56.292 DEBUG [src.node.waku_node] Starting Node...
2026-03-12 04:35:56.292 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-12 04:35:56.294 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-12 04:35:56.294 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.98.239
2026-03-12 04:35:56.294 DEBUG [src.node.docker_mananger] Generated ports ['63900', '63901', '63902', '63903', '63904']
2026-03-12 04:35:56.294 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-12 04:35:56.294 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-12 04:35:56.295 DEBUG [src.node.waku_node] Using volumes []
2026-03-12 04:35:56.295 DEBUG [src.node.docker_mananger] docker run -i -t -p 63900:63900 -p 63901:63901 -p 63902:63902 -p 63903:63903 -p 63904:63904 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=63902 --rest-port=63900 --tcp-port=63901 --discv5-udp-port=63903 --rest-address=0.0.0.0 --nat=extip:172.18.98.239 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=e2f0a340ca8d00c1cf29dbbf81d30cb6ca40bd4cf7e92e35d686fec3034cbf2c --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=63904 --metrics-logging=true --relay=true
2026-03-12 04:35:56.477 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.98.239 waku ff582e1f31055ba67ce208ff43fb0e67699309f8e955c2d2028e30d8259f1bba
2026-03-12 04:35:56.515 DEBUG [src.node.docker_mananger] Container started with ID ff582e1f3105. Setting up logs at ./log/docker/node1_2026-03-12_04-35-56__6621863f-c62c-4624-91a7-258290dc259a__wakuorg_nwaku:latest.log
2026-03-12 04:35:56.516 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 63900
2026-03-12 04:35:56.516 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-12 04:35:56.519 ERROR [src.node.docker_mananger] Max retries reached for container 3fbc907caabf. Exiting log stream.
2026-03-12 04:35:57.061 ERROR [src.node.docker_mananger] Max retries reached for container f8dde3cbf9cd. Exiting log stream.
2026-03-12 04:35:57.516 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63900/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:57.519 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-12 04:35:57.519 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-12 04:35:57.519 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63900/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:57.522 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.98.239/tcp/63901/p2p/16Uiu2HAm2wmXr81cmceemF3284Kn8Jq824mKxsxaobsTf7MujY9W","/ip4/172.18.98.239/tcp/63902/ws/p2p/16Uiu2HAm2wmXr81cmceemF3284Kn8Jq824mKxsxaobsTf7MujY9W"],"enrUri":"enr:-L24QFRH5E4oeHSZxqXA59yln5wmhROzvfS9zcdjXYetLl8IeGf2L4ILvuodsvnMIqO86zFOEpBw0XNL6qW2RdQVNx0CgmlkgnY0gmlwhKwSYu-KbXVsdGlhZGRyc5YACASsEmLvBvmdAAoErBJi7wb5nt0DgnJzhQADAQAAiXNlY3AyNTZrMaECb7eC2-jKHnTgCKhXjUrUpikp8SYA6aXxdxaQxQhjxhmDdGNwgvmdg3VkcIL5n4V3YWt1MgE"}'
2026-03-12 04:35:57.522 INFO [src.node.waku_node] REST service is ready !!
2026-03-12 04:35:57.522 DEBUG [src.node.waku_node] Starting Node...
2026-03-12 04:35:57.522 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-12 04:35:57.523 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-12 04:35:57.524 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.35.26
2026-03-12 04:35:57.524 DEBUG [src.node.docker_mananger] Generated ports ['11420', '11421', '11422', '11423', '11424']
2026-03-12 04:35:57.524 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-12 04:35:57.524 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-12 04:35:57.524 DEBUG [src.node.waku_node] Using volumes []
2026-03-12 04:35:57.524 DEBUG [src.node.docker_mananger] docker run -i -t -p 11420:11420 -p 11421:11421 -p 11422:11422 -p 11423:11423 -p 11424:11424 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=11422 --rest-port=11420 --tcp-port=11421 --discv5-udp-port=11423 --rest-address=0.0.0.0 --nat=extip:172.18.35.26 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=a29bc913c095edec01672a1122bdd6d75cef55ec6bd3da6be5e572d7f4f58e51 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=11424 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QFRH5E4oeHSZxqXA59yln5wmhROzvfS9zcdjXYetLl8IeGf2L4ILvuodsvnMIqO86zFOEpBw0XNL6qW2RdQVNx0CgmlkgnY0gmlwhKwSYu-KbXVsdGlhZGRyc5YACASsEmLvBvmdAAoErBJi7wb5nt0DgnJzhQADAQAAiXNlY3AyNTZrMaECb7eC2-jKHnTgCKhXjUrUpikp8SYA6aXxdxaQxQhjxhmDdGNwgvmdg3VkcIL5n4V3YWt1MgE
2026-03-12 04:35:57.711 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.35.26 waku 3941464409f9b659416362cb1862ae2bf5f2d9d31db7ce56f9bbd127561f9152
2026-03-12 04:35:57.746 DEBUG [src.node.docker_mananger] Container started with ID 3941464409f9. Setting up logs at ./log/docker/node2_2026-03-12_04-35-56__6621863f-c62c-4624-91a7-258290dc259a__wakuorg_nwaku:latest.log
2026-03-12 04:35:57.747 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 11420
2026-03-12 04:35:57.747 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-12 04:35:58.750 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:11420/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:58.772 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-12 04:35:58.775 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-12 04:35:58.776 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:11420/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:58.785 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.35.26/tcp/11421/p2p/16Uiu2HAmUHPaNgL1C5WSNp1fug4egeZPcHw9Jr2Sza2WJArp5Z15","/ip4/172.18.35.26/tcp/11422/ws/p2p/16Uiu2HAmUHPaNgL1C5WSNp1fug4egeZPcHw9Jr2Sza2WJArp5Z15"],"enrUri":"enr:-L24QIOgTg3AyIH07nXaIvmmt_kP_-IVSZ6w57OyqncmsyYCPsgtKEcDKtVcHeX1V9JV_w6Df4fK6CWYzDu9bznOGEQCgmlkgnY0gmlwhKwSIxqKbXVsdGlhZGRyc5YACASsEiMaBiydAAoErBIjGgYsnt0DgnJzhQADAQAAiXNlY3AyNTZrMaED6DJ5awxtTaAGj371KqxWCTvac1SCEd9X_k-2a7YE9HSDdGNwgiydg3VkcIIsn4V3YWt1MgE"}'
2026-03-12 04:35:58.785 INFO [src.node.waku_node] REST service is ready !!
2026-03-12 04:35:58.786 DEBUG [src.node.waku_node] Starting Node...
2026-03-12 04:35:58.786 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-12 04:35:58.791 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-12 04:35:58.791 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.85.121
2026-03-12 04:35:58.791 DEBUG [src.node.docker_mananger] Generated ports ['42386', '42387', '42388', '42389', '42390']
2026-03-12 04:35:58.791 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-12 04:35:58.792 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-12 04:35:58.792 DEBUG [src.node.waku_node] Using volumes []
2026-03-12 04:35:58.792 DEBUG [src.node.docker_mananger] docker run -i -t -p 42386:42386 -p 42387:42387 -p 42388:42388 -p 42389:42389 -p 42390:42390 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=42388 --rest-port=42386 --tcp-port=42387 --discv5-udp-port=42389 --rest-address=0.0.0.0 --nat=extip:172.18.85.121 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=d9daaa4d61483e44f805d01d6760fd5aa5b8de3feeeeef30fac43d48c14fb4e9 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=42390 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QIOgTg3AyIH07nXaIvmmt_kP_-IVSZ6w57OyqncmsyYCPsgtKEcDKtVcHeX1V9JV_w6Df4fK6CWYzDu9bznOGEQCgmlkgnY0gmlwhKwSIxqKbXVsdGlhZGRyc5YACASsEiMaBiydAAoErBIjGgYsnt0DgnJzhQADAQAAiXNlY3AyNTZrMaED6DJ5awxtTaAGj371KqxWCTvac1SCEd9X_k-2a7YE9HSDdGNwgiydg3VkcIIsn4V3YWt1MgE
2026-03-12 04:35:58.998 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.85.121 waku 8439c36af7e8668f422be5e7124ae85e906806bb6d12b713478b9c157c2b160b
2026-03-12 04:35:59.033 DEBUG [src.node.docker_mananger] Container started with ID 8439c36af7e8. Setting up logs at ./log/docker/node3_2026-03-12_04-35-56__6621863f-c62c-4624-91a7-258290dc259a__wakuorg_nwaku:latest.log
2026-03-12 04:35:59.034 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 42386
2026-03-12 04:35:59.034 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-12 04:36:00.036 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42386/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:36:00.049 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-12 04:36:00.050 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-12 04:36:00.051 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42386/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:36:00.058 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.85.121/tcp/42387/p2p/16Uiu2HAmACcxxYAw7Abqgf7ho6Mw2NPZrutCxMRez5QdLcz1ioY6","/ip4/172.18.85.121/tcp/42388/ws/p2p/16Uiu2HAmACcxxYAw7Abqgf7ho6Mw2NPZrutCxMRez5QdLcz1ioY6"],"enrUri":"enr:-L24QKOnDimFR-vIuBFe_uIUENHKffvge4NdRMAAUZkUehlnQl6pPiD171xiDRyuhqTkcE6WcSIxYYdVX3V59vLO65ICgmlkgnY0gmlwhKwSVXmKbXVsdGlhZGRyc5YACASsElV5BqWTAAoErBJVeQallN0DgnJzhQADAQAAiXNlY3AyNTZrMaEC24dYUPK4NBxKsMQTc3apzM2r2UQzO-CujDxk2pYzKMuDdGNwgqWTg3VkcIKllYV3YWt1MgE"}'
2026-03-12 04:36:00.059 INFO [src.node.waku_node] REST service is ready !!
2026-03-12 04:36:00.059 DEBUG [src.node.waku_node] Starting Node...
2026-03-12 04:36:00.060 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-12 04:36:00.062 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-12 04:36:00.062 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.49.136
2026-03-12 04:36:00.062 DEBUG [src.node.docker_mananger] Generated ports ['40311', '40312', '40313', '40314', '40315']
2026-03-12 04:36:00.062 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-12 04:36:00.063 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-12 04:36:00.063 DEBUG [src.node.waku_node] Using volumes []
2026-03-12 04:36:00.063 DEBUG [src.node.docker_mananger] docker run -i -t -p 40311:40311 -p 40312:40312 -p 40313:40313 -p 40314:40314 -p 40315:40315 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=40313 --rest-port=40311 --tcp-port=40312 --discv5-udp-port=40314 --rest-address=0.0.0.0 --nat=extip:172.18.49.136 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=ff0cfc380c3ffb32edab32ba8fbbf8ec1d238d5fa2b7adb80cd0ef865a9372c4 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=40315 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QKOnDimFR-vIuBFe_uIUENHKffvge4NdRMAAUZkUehlnQl6pPiD171xiDRyuhqTkcE6WcSIxYYdVX3V59vLO65ICgmlkgnY0gmlwhKwSVXmKbXVsdGlhZGRyc5YACASsElV5BqWTAAoErBJVeQallN0DgnJzhQADAQAAiXNlY3AyNTZrMaEC24dYUPK4NBxKsMQTc3apzM2r2UQzO-CujDxk2pYzKMuDdGNwgqWTg3VkcIKllYV3YWt1MgE
2026-03-12 04:36:00.255 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.49.136 waku b9c2baae7e4a2406536abd24d68eca9706a58c3b09196ac1a9f6df9ff01e5e91
2026-03-12 04:36:00.291 DEBUG [src.node.docker_mananger] Container started with ID b9c2baae7e4a. Setting up logs at ./log/docker/node4_2026-03-12_04-35-56__6621863f-c62c-4624-91a7-258290dc259a__wakuorg_nwaku:latest.log
2026-03-12 04:36:00.291 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 40311
2026-03-12 04:36:00.291 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-12 04:36:01.293 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:40311/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:36:01.308 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-12 04:36:01.311 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-12 04:36:01.312 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:40311/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:36:01.318 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.49.136/tcp/40312/p2p/16Uiu2HAmVT4kTvsCsyUff5t8BKyAkc85toDdvEgrHviWqjtsfZ3Y","/ip4/172.18.49.136/tcp/40313/ws/p2p/16Uiu2HAmVT4kTvsCsyUff5t8BKyAkc85toDdvEgrHviWqjtsfZ3Y"],"enrUri":"enr:-L24QHHr9gkH8kVcK1aG0YbNqk-6MN8ogKs31rkjBMUmjep6K2THx7fqMifI9VrXKau1iqz9HwI7lmmtr8Q045skCdUCgmlkgnY0gmlwhKwSMYiKbXVsdGlhZGRyc5YACASsEjGIBp14AAoErBIxiAaded0DgnJzhQADAQAAiXNlY3AyNTZrMaED-YiuwkikAI2s3MDNWV-PJ_fJPbZyPnNU3vASpV-47IODdGNwgp14g3VkcIKdeoV3YWt1MgE"}'
2026-03-12 04:36:01.318 INFO [src.node.waku_node] REST service is ready !!
2026-03-12 04:36:01.319 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63900/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-12 04:36:01.323 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:36:01.324 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:11420/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-12 04:36:01.329 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:36:01.332 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:42386/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-12 04:36:01.340 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:36:01.341 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40311/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-12 04:36:01.347 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:36:01.348 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63900/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:36:01.352 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.35.26/tcp/42336/p2p/16Uiu2HAmUHPaNgL1C5WSNp1fug4egeZPcHw9Jr2Sza2WJArp5Z15","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"UnknownOrigin"}]'
2026-03-12 04:36:01.352 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:11420/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:36:01.354 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.85.121/tcp/60212/p2p/16Uiu2HAmACcxxYAw7Abqgf7ho6Mw2NPZrutCxMRez5QdLcz1ioY6","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.98.239/tcp/63901/p2p/16Uiu2HAm2wmXr81cmceemF3284Kn8Jq824mKxsxaobsTf7MujY9W","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"Discv5"}]'
2026-03-12 04:36:01.354 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42386/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:36:01.357 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.35.26/tcp/11421/p2p/16Uiu2HAmUHPaNgL1C5WSNp1fug4egeZPcHw9Jr2Sza2WJArp5Z15","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"Discv5"},{"multiaddr":"/ip4/172.18.49.136/tcp/46910/p2p/16Uiu2HAmVT4kTvsCsyUff5t8BKyAkc85toDdvEgrHviWqjtsfZ3Y","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"UnknownOrigin"}]'
2026-03-12 04:36:01.357 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:40311/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:36:01.359 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.85.121/tcp/42387/p2p/16Uiu2HAmACcxxYAw7Abqgf7ho6Mw2NPZrutCxMRez5QdLcz1ioY6","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"Discv5"}]'
2026-03-12 04:36:01.359 DEBUG [src.libs.common] Sleeping for 20 seconds
2026-03-12 04:36:21.361 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7661', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-12 04:36:21.372 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7661', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'loss', '50.0%']
2026-03-12 04:36:21.382 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7661', '-n', 'tc', '-s', 'qdisc', 'show', 'dev', 'eth0']
2026-03-12 04:36:21.391 DEBUG [src.steps.network_conditions] tc -s qdisc show dev eth0:
qdisc netem 8002: root refcnt 5 limit 1000 loss 50%
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
2026-03-12 04:36:21.392 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:40311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:36:21.394 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[]'
2026-03-12 04:36:21.394 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63900/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-12 04:36:22.618 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:36:22.618 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63900/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-12 04:36:23.631 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:36:23.632 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63900/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-12 04:36:24.858 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:36:24.858 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63900/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-12 04:36:25.873 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:36:25.874 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63900/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-12 04:36:26.506 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:36:26.508 DEBUG [src.libs.common] Sleeping for 30.0 seconds
2026-03-12 04:36:56.509 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:40311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:36:56.518 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773290181394782272,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773290182618846182,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773290183632575075,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773290184858448386,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773290185873763948,"ephemeral":false,"proof":""}]'
2026-03-12 04:36:56.529 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7661', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-12 04:36:56.545 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7661', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-12 04:36:56.556 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7661', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'loss', '50.0%', '75.0%']
2026-03-12 04:36:56.566 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:40311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:36:56.568 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[]'
2026-03-12 04:36:56.568 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63900/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-12 04:37:00.482 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:37:00.483 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63900/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-12 04:37:00.489 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:37:00.490 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63900/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-12 04:37:00.510 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:37:00.510 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63900/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-12 04:37:00.529 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:37:00.531 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63900/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-12 04:37:00.547 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:37:00.547 DEBUG [src.libs.common] Sleeping for 30.0 seconds
2026-03-12 04:37:30.547 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:40311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:37:30.550 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773290216568781982,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773290220482939818,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773290220490173886,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773290220510569560,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773290220531647851,"ephemeral":false,"proof":""}]'
2026-03-12 04:37:30.551 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7661', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-12 04:37:30.562 DEBUG [tests.e2e.test_network_conditions] uncorrelated=5 correlated=5
2026-03-12 04:37:30.563 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-12 04:37:30.564 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-12 04:37:30.564 DEBUG [src.node.waku_node] Stopping container with id ff582e1f3105
2026-03-12 04:37:31.106 DEBUG [src.node.waku_node] Container stopped.
2026-03-12 04:37:31.106 DEBUG [src.node.waku_node] Stopping container with id 3941464409f9
2026-03-12 04:37:31.691 DEBUG [src.node.waku_node] Container stopped.
2026-03-12 04:37:31.694 DEBUG [src.node.waku_node] Stopping container with id 8439c36af7e8
2026-03-12 04:37:31.920 ERROR [src.node.docker_mananger] Max retries reached for container ff582e1f3105. Exiting log stream.
2026-03-12 04:37:32.216 DEBUG [src.node.waku_node] Container stopped.
2026-03-12 04:37:32.218 DEBUG [src.node.waku_node] Stopping container with id b9c2baae7e4a
2026-03-12 04:37:32.504 ERROR [src.node.docker_mananger] Max retries reached for container 3941464409f9. Exiting log stream.
2026-03-12 04:37:32.794 DEBUG [src.node.waku_node] Container stopped.
2026-03-12 04:37:32.795 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-12 04:37:32.866 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-12 04:37:32.942 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-12 04:37:33.013 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-12 04:37:33.086 DEBUG [src.node.docker_mananger] No errors found in the waku logs.