169 lines
34 KiB
Plaintext

2026-03-02 04:36:30.171 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-02 04:36:30.172 DEBUG [tests.conftest] Running test: test_relay_packet_loss_correlated_vs_uncorrelated with id: 2026-03-02_04-36-30__054c243c-680b-4e3b-9395-1b04c24ed8d9
2026-03-02 04:36:30.172 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-02 04:36:30.172 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-02 04:36:30.179 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-02 04:36:30.179 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-02_04-36-30__054c243c-680b-4e3b-9395-1b04c24ed8d9__wakuorg_nwaku:latest.log
2026-03-02 04:36:30.185 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-02 04:36:30.185 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-02_04-36-30__054c243c-680b-4e3b-9395-1b04c24ed8d9__wakuorg_nwaku:latest.log
2026-03-02 04:36:30.192 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-02 04:36:30.192 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-02_04-36-30__054c243c-680b-4e3b-9395-1b04c24ed8d9__wakuorg_nwaku:latest.log
2026-03-02 04:36:30.198 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-02 04:36:30.199 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-02_04-36-30__054c243c-680b-4e3b-9395-1b04c24ed8d9__wakuorg_nwaku:latest.log
2026-03-02 04:36:30.199 DEBUG [src.node.waku_node] Starting Node...
2026-03-02 04:36:30.199 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-02 04:36:30.200 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-02 04:36:30.200 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.243.179
2026-03-02 04:36:30.201 DEBUG [src.node.docker_mananger] Generated ports ['58233', '58234', '58235', '58236', '58237']
2026-03-02 04:36:30.201 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-02 04:36:30.201 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-02 04:36:30.201 DEBUG [src.node.waku_node] Using volumes []
2026-03-02 04:36:30.201 DEBUG [src.node.docker_mananger] docker run -i -t -p 58233:58233 -p 58234:58234 -p 58235:58235 -p 58236:58236 -p 58237:58237 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=58235 --rest-port=58233 --tcp-port=58234 --discv5-udp-port=58236 --rest-address=0.0.0.0 --nat=extip:172.18.243.179 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=23b18ff60be75319c1afed18fbefb89607b14084ff7cb6bece3d2bf7cd00e36f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=58237 --metrics-logging=true --relay=true
2026-03-02 04:36:30.397 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.243.179 waku eb5668150cc96887c202d4fb77cefaa1754b5d790a1555156490e898ab3c8934
2026-03-02 04:36:30.401 ERROR [src.node.docker_mananger] Max retries reached for container 8b8109a58793. Exiting log stream.
2026-03-02 04:36:30.433 DEBUG [src.node.docker_mananger] Container started with ID eb5668150cc9. Setting up logs at ./log/docker/node1_2026-03-02_04-36-30__054c243c-680b-4e3b-9395-1b04c24ed8d9__wakuorg_nwaku:latest.log
2026-03-02 04:36:30.433 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 58233
2026-03-02 04:36:30.433 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-02 04:36:30.954 ERROR [src.node.docker_mananger] Max retries reached for container b0820ad62b00. Exiting log stream.
2026-03-02 04:36:31.435 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58233/health" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:31.437 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-02 04:36:31.438 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-02 04:36:31.438 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58233/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:31.440 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.243.179/tcp/58234/p2p/16Uiu2HAkxwEX2B1vYqk4HX3r5istjhfdkcyv6hG3qUnGYsnNjN4B","/ip4/172.18.243.179/tcp/58235/ws/p2p/16Uiu2HAkxwEX2B1vYqk4HX3r5istjhfdkcyv6hG3qUnGYsnNjN4B"],"enrUri":"enr:-L24QL1lK7YgN_MH4fyukAC68pSkNU0ryVC_OE3pjItaxYNfJR7VIKMmIEKyA7-jyD2ljY3ebDMFDq--TNKZnd1TKkQCgmlkgnY0gmlwhKwS87OKbXVsdGlhZGRyc5YACASsEvOzBuN6AAoErBLzswbje90DgnJzhQADAQAAiXNlY3AyNTZrMaECNCWyDo5aBRXvwvaBVsKAcDstN_xKFPyT_JEFIIJxiayDdGNwguN6g3VkcILjfIV3YWt1MgE"}'
2026-03-02 04:36:31.440 INFO [src.node.waku_node] REST service is ready !!
2026-03-02 04:36:31.440 DEBUG [src.node.waku_node] Starting Node...
2026-03-02 04:36:31.441 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-02 04:36:31.442 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-02 04:36:31.442 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.235.229
2026-03-02 04:36:31.442 DEBUG [src.node.docker_mananger] Generated ports ['5256', '5257', '5258', '5259', '5260']
2026-03-02 04:36:31.442 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-02 04:36:31.443 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-02 04:36:31.443 DEBUG [src.node.waku_node] Using volumes []
2026-03-02 04:36:31.443 DEBUG [src.node.docker_mananger] docker run -i -t -p 5256:5256 -p 5257:5257 -p 5258:5258 -p 5259:5259 -p 5260:5260 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=5258 --rest-port=5256 --tcp-port=5257 --discv5-udp-port=5259 --rest-address=0.0.0.0 --nat=extip:172.18.235.229 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=46d0c5cd1b9bd2bb32a17eaa0fe8cbaf1e3d7f55f609be618e654799c8e4b82f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=5260 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QL1lK7YgN_MH4fyukAC68pSkNU0ryVC_OE3pjItaxYNfJR7VIKMmIEKyA7-jyD2ljY3ebDMFDq--TNKZnd1TKkQCgmlkgnY0gmlwhKwS87OKbXVsdGlhZGRyc5YACASsEvOzBuN6AAoErBLzswbje90DgnJzhQADAQAAiXNlY3AyNTZrMaECNCWyDo5aBRXvwvaBVsKAcDstN_xKFPyT_JEFIIJxiayDdGNwguN6g3VkcILjfIV3YWt1MgE
2026-03-02 04:36:31.637 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.235.229 waku e35466262bf709aba4f9df864bb758556e5bf6586ed57eb75cbf9a0defeb8d58
2026-03-02 04:36:31.672 DEBUG [src.node.docker_mananger] Container started with ID e35466262bf7. Setting up logs at ./log/docker/node2_2026-03-02_04-36-30__054c243c-680b-4e3b-9395-1b04c24ed8d9__wakuorg_nwaku:latest.log
2026-03-02 04:36:31.672 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 5256
2026-03-02 04:36:31.672 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-02 04:36:32.673 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5256/health" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:32.698 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-02 04:36:32.700 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-02 04:36:32.701 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5256/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:32.706 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.235.229/tcp/5257/p2p/16Uiu2HAkw85LavmCiAtd1oeqgCGqMc3LxaYdi1DxEpNrw8US9aGt","/ip4/172.18.235.229/tcp/5258/ws/p2p/16Uiu2HAkw85LavmCiAtd1oeqgCGqMc3LxaYdi1DxEpNrw8US9aGt"],"enrUri":"enr:-L24QOcF0PFxc26sDoRZj1zciVipI2bxXowLbKwMApAftW-7IwBKXFZTRSW37oHK45UN6I8EJWdg4Qis_cCYfWA4-LgCgmlkgnY0gmlwhKwS6-WKbXVsdGlhZGRyc5YACASsEuvlBhSJAAoErBLr5QYUit0DgnJzhQADAQAAiXNlY3AyNTZrMaECGTVUsEeeyUKYRawlUwL20zY82KV_oxuG6WChRDVbfq2DdGNwghSJg3VkcIIUi4V3YWt1MgE"}'
2026-03-02 04:36:32.708 INFO [src.node.waku_node] REST service is ready !!
2026-03-02 04:36:32.708 DEBUG [src.node.waku_node] Starting Node...
2026-03-02 04:36:32.708 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-02 04:36:32.710 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-02 04:36:32.710 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.63.215
2026-03-02 04:36:32.710 DEBUG [src.node.docker_mananger] Generated ports ['13228', '13229', '13230', '13231', '13232']
2026-03-02 04:36:32.710 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-02 04:36:32.710 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-02 04:36:32.710 DEBUG [src.node.waku_node] Using volumes []
2026-03-02 04:36:32.711 DEBUG [src.node.docker_mananger] docker run -i -t -p 13228:13228 -p 13229:13229 -p 13230:13230 -p 13231:13231 -p 13232:13232 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=13230 --rest-port=13228 --tcp-port=13229 --discv5-udp-port=13231 --rest-address=0.0.0.0 --nat=extip:172.18.63.215 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=1c7a7ebcd9f2dc40bd7ae8f6b2b6afebfe002c32cfe0d8b5adff3e9ddadac1e4 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=13232 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QOcF0PFxc26sDoRZj1zciVipI2bxXowLbKwMApAftW-7IwBKXFZTRSW37oHK45UN6I8EJWdg4Qis_cCYfWA4-LgCgmlkgnY0gmlwhKwS6-WKbXVsdGlhZGRyc5YACASsEuvlBhSJAAoErBLr5QYUit0DgnJzhQADAQAAiXNlY3AyNTZrMaECGTVUsEeeyUKYRawlUwL20zY82KV_oxuG6WChRDVbfq2DdGNwghSJg3VkcIIUi4V3YWt1MgE
2026-03-02 04:36:32.905 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.63.215 waku d143e05f44e12ebbddaee0f1f1e71235ff008c50712de02a00662300fecb86b4
2026-03-02 04:36:32.941 DEBUG [src.node.docker_mananger] Container started with ID d143e05f44e1. Setting up logs at ./log/docker/node3_2026-03-02_04-36-30__054c243c-680b-4e3b-9395-1b04c24ed8d9__wakuorg_nwaku:latest.log
2026-03-02 04:36:32.941 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 13228
2026-03-02 04:36:32.942 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-02 04:36:33.943 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13228/health" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:33.959 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-02 04:36:33.959 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-02 04:36:33.960 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13228/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:33.967 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.63.215/tcp/13229/p2p/16Uiu2HAkw3Jwd8j6rpqy28eYPCK6R6GgMaoYxthpRE3bEXQ3wS7G","/ip4/172.18.63.215/tcp/13230/ws/p2p/16Uiu2HAkw3Jwd8j6rpqy28eYPCK6R6GgMaoYxthpRE3bEXQ3wS7G"],"enrUri":"enr:-L24QJSfqVw5-dyuK6uCY9qJ2mQFgukwEzqsiyFstpdqyFq0dtxu_seT8yBbKqSLhQGbmHbFavUMEQLRCEY97yVM_VYCgmlkgnY0gmlwhKwSP9eKbXVsdGlhZGRyc5YACASsEj_XBjOtAAoErBI_1wYzrt0DgnJzhQADAQAAiXNlY3AyNTZrMaECF_zPGT-hz1z3L44Xh_IXhjkazzW4GYDEPoDsjnhH99-DdGNwgjOtg3VkcIIzr4V3YWt1MgE"}'
2026-03-02 04:36:33.967 INFO [src.node.waku_node] REST service is ready !!
2026-03-02 04:36:33.967 DEBUG [src.node.waku_node] Starting Node...
2026-03-02 04:36:33.967 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-02 04:36:33.969 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-02 04:36:33.969 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.62.13
2026-03-02 04:36:33.970 DEBUG [src.node.docker_mananger] Generated ports ['30864', '30865', '30866', '30867', '30868']
2026-03-02 04:36:33.970 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-02 04:36:33.970 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-02 04:36:33.970 DEBUG [src.node.waku_node] Using volumes []
2026-03-02 04:36:33.970 DEBUG [src.node.docker_mananger] docker run -i -t -p 30864:30864 -p 30865:30865 -p 30866:30866 -p 30867:30867 -p 30868:30868 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=30866 --rest-port=30864 --tcp-port=30865 --discv5-udp-port=30867 --rest-address=0.0.0.0 --nat=extip:172.18.62.13 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=d1c3a6eecdab5ebd97e02b353acf36ae6071a589edaa5bb847a4f57eccac11d7 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=30868 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QJSfqVw5-dyuK6uCY9qJ2mQFgukwEzqsiyFstpdqyFq0dtxu_seT8yBbKqSLhQGbmHbFavUMEQLRCEY97yVM_VYCgmlkgnY0gmlwhKwSP9eKbXVsdGlhZGRyc5YACASsEj_XBjOtAAoErBI_1wYzrt0DgnJzhQADAQAAiXNlY3AyNTZrMaECF_zPGT-hz1z3L44Xh_IXhjkazzW4GYDEPoDsjnhH99-DdGNwgjOtg3VkcIIzr4V3YWt1MgE
2026-03-02 04:36:34.167 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.62.13 waku 7a75026f6337a71aee9258d262645b9b46ea53011be65281066de6688bfca8c4
2026-03-02 04:36:34.199 DEBUG [src.node.docker_mananger] Container started with ID 7a75026f6337. Setting up logs at ./log/docker/node4_2026-03-02_04-36-30__054c243c-680b-4e3b-9395-1b04c24ed8d9__wakuorg_nwaku:latest.log
2026-03-02 04:36:34.200 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 30864
2026-03-02 04:36:34.201 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-02 04:36:35.202 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30864/health" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:35.220 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-02 04:36:35.220 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-02 04:36:35.220 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30864/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:35.227 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.62.13/tcp/30865/p2p/16Uiu2HAmSEwBi3JashRv2tqvSPR4b19D5nUXx23Em9Pm1LiqKZdB","/ip4/172.18.62.13/tcp/30866/ws/p2p/16Uiu2HAmSEwBi3JashRv2tqvSPR4b19D5nUXx23Em9Pm1LiqKZdB"],"enrUri":"enr:-L24QBrEQdXmdgxFQV-6iOUwwEKrwgrDvx_vJhVdTfAnSHaqCjuxOHL9pq-CnQnUA5fy4frk6kWTO3aw6p-ADlFtAqACgmlkgnY0gmlwhKwSPg2KbXVsdGlhZGRyc5YACASsEj4NBniRAAoErBI-DQZ4kt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDydoZQSfy24dOQi1JwqcB72Ovm5-BR3aUo3C-EASVG-KDdGNwgniRg3VkcIJ4k4V3YWt1MgE"}'
2026-03-02 04:36:35.227 INFO [src.node.waku_node] REST service is ready !!
2026-03-02 04:36:35.228 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58233/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-02 04:36:35.233 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:36:35.234 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:5256/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-02 04:36:35.240 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:36:35.242 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13228/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-02 04:36:35.252 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:36:35.252 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30864/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-02 04:36:35.259 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:36:35.263 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58233/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:35.266 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.235.229/tcp/52132/p2p/16Uiu2HAkw85LavmCiAtd1oeqgCGqMc3LxaYdi1DxEpNrw8US9aGt","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-157-gba8587","origin":"UnknownOrigin"}]'
2026-03-02 04:36:35.266 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5256/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:35.269 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.63.215/tcp/59728/p2p/16Uiu2HAkw3Jwd8j6rpqy28eYPCK6R6GgMaoYxthpRE3bEXQ3wS7G","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-157-gba8587","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.243.179/tcp/58234/p2p/16Uiu2HAkxwEX2B1vYqk4HX3r5istjhfdkcyv6hG3qUnGYsnNjN4B","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-157-gba8587","origin":"Discv5"}]'
2026-03-02 04:36:35.269 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13228/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:35.271 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.235.229/tcp/5257/p2p/16Uiu2HAkw85LavmCiAtd1oeqgCGqMc3LxaYdi1DxEpNrw8US9aGt","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-157-gba8587","origin":"Discv5"},{"multiaddr":"/ip4/172.18.62.13/tcp/48154/p2p/16Uiu2HAmSEwBi3JashRv2tqvSPR4b19D5nUXx23Em9Pm1LiqKZdB","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-157-gba8587","origin":"UnknownOrigin"}]'
2026-03-02 04:36:35.272 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30864/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:35.274 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.63.215/tcp/13229/p2p/16Uiu2HAkw3Jwd8j6rpqy28eYPCK6R6GgMaoYxthpRE3bEXQ3wS7G","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-157-gba8587","origin":"Discv5"}]'
2026-03-02 04:36:35.274 DEBUG [src.libs.common] Sleeping for 20 seconds
2026-03-02 04:36:55.276 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7693', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-02 04:36:55.287 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7693', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'loss', '50.0%']
2026-03-02 04:36:55.298 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7693', '-n', 'tc', '-s', 'qdisc', 'show', 'dev', 'eth0']
2026-03-02 04:36:55.307 DEBUG [src.steps.network_conditions] tc -s qdisc show dev eth0:
qdisc netem 8002: root refcnt 5 limit 1000 loss 50%
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
2026-03-02 04:36:55.307 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30864/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:36:55.309 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[]'
2026-03-02 04:36:55.309 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58233/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-02 04:36:55.527 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:36:55.528 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58233/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-02 04:36:56.550 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:36:56.553 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58233/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-02 04:36:57.574 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:36:57.579 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58233/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-02 04:37:00.032 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:37:00.032 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58233/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-02 04:37:00.040 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:37:00.043 DEBUG [src.libs.common] Sleeping for 30.0 seconds
2026-03-02 04:37:30.043 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30864/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:37:30.047 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426215309813438,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426215528249619,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426216552898411,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426217579804870,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426220032557186,"ephemeral":false,"proof":""}]'
2026-03-02 04:37:30.049 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7693', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-02 04:37:30.060 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7693', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-02 04:37:30.072 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7693', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'loss', '50.0%', '75.0%']
2026-03-02 04:37:30.082 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30864/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:37:30.085 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[]'
2026-03-02 04:37:30.086 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58233/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-02 04:37:30.093 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:37:30.095 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58233/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-02 04:37:30.108 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:37:30.111 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58233/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-02 04:37:32.832 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:37:32.832 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58233/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-02 04:37:32.841 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:37:32.844 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58233/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-02 04:37:33.866 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:37:33.866 DEBUG [src.libs.common] Sleeping for 30.0 seconds
2026-03-02 04:38:03.868 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30864/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:38:03.871 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426250085927410,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426250094931097,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426250111540692,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426252832570305,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426252844046040,"ephemeral":false,"proof":""}]'
2026-03-02 04:38:03.873 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '7693', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-02 04:38:03.883 DEBUG [tests.e2e.test_network_conditions] uncorrelated=5 correlated=5
2026-03-02 04:38:03.885 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-02 04:38:03.886 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-02 04:38:03.886 DEBUG [src.node.waku_node] Stopping container with id eb5668150cc9
2026-03-02 04:38:04.460 DEBUG [src.node.waku_node] Container stopped.
2026-03-02 04:38:04.461 DEBUG [src.node.waku_node] Stopping container with id e35466262bf7
2026-03-02 04:38:05.045 DEBUG [src.node.waku_node] Container stopped.
2026-03-02 04:38:05.049 DEBUG [src.node.waku_node] Stopping container with id d143e05f44e1
2026-03-02 04:38:05.285 ERROR [src.node.docker_mananger] Max retries reached for container eb5668150cc9. Exiting log stream.
2026-03-02 04:38:05.615 DEBUG [src.node.waku_node] Container stopped.
2026-03-02 04:38:05.615 DEBUG [src.node.waku_node] Stopping container with id 7a75026f6337
2026-03-02 04:38:06.137 ERROR [src.node.docker_mananger] Max retries reached for container e35466262bf7. Exiting log stream.
2026-03-02 04:38:06.188 DEBUG [src.node.waku_node] Container stopped.
2026-03-02 04:38:06.189 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-02 04:38:06.270 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-02 04:38:06.423 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-02 04:38:06.496 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-02 04:38:06.567 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-02 04:38:06.571 ERROR [src.node.docker_mananger] Max retries reached for container d143e05f44e1. Exiting log stream.