238 lines
47 KiB
Plaintext

2026-03-22 04:39:01.342 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-22 04:39:01.343 DEBUG [tests.conftest] Running test: test_latency_with_load_sender_side with id: 2026-03-22_04-39-01__3c67be1d-c4a3-48d0-9da9-e022f94d1906
2026-03-22 04:39:01.343 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-22 04:39:01.343 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-22 04:39:01.350 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-22 04:39:01.350 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-22_04-39-01__3c67be1d-c4a3-48d0-9da9-e022f94d1906__wakuorg_nwaku:latest.log
2026-03-22 04:39:01.357 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-22 04:39:01.357 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-22_04-39-01__3c67be1d-c4a3-48d0-9da9-e022f94d1906__wakuorg_nwaku:latest.log
2026-03-22 04:39:01.364 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-22 04:39:01.364 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-22_04-39-01__3c67be1d-c4a3-48d0-9da9-e022f94d1906__wakuorg_nwaku:latest.log
2026-03-22 04:39:01.371 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-22 04:39:01.371 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-22_04-39-01__3c67be1d-c4a3-48d0-9da9-e022f94d1906__wakuorg_nwaku:latest.log
2026-03-22 04:39:01.371 INFO [tests.e2e.test_network_conditions] Starting 4 nodes with relay enabled
2026-03-22 04:39:01.371 DEBUG [src.node.waku_node] Starting Node...
2026-03-22 04:39:01.371 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-22 04:39:01.373 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-22 04:39:01.373 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.52.0
2026-03-22 04:39:01.373 DEBUG [src.node.docker_mananger] Generated ports ['60948', '60949', '60950', '60951', '60952']
2026-03-22 04:39:01.373 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-22 04:39:01.373 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-22 04:39:01.373 DEBUG [src.node.waku_node] Using volumes []
2026-03-22 04:39:01.374 DEBUG [src.node.docker_mananger] docker run -i -t -p 60948:60948 -p 60949:60949 -p 60950:60950 -p 60951:60951 -p 60952:60952 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=60950 --rest-port=60948 --tcp-port=60949 --discv5-udp-port=60951 --rest-address=0.0.0.0 --nat=extip:172.18.52.0 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=bff699e67fe3da4bcd81c9a2bc6ba7988b22c2cfdcbc5ceeb846d1ed5d8eb81c --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=60952 --metrics-logging=true --relay=true
2026-03-22 04:39:01.552 ERROR [src.node.docker_mananger] Max retries reached for container ef8f1643508c. Exiting log stream.
2026-03-22 04:39:01.584 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.52.0 waku 1eb271796e5750762cc91ebdd6c8d01953280ee7dc607b05a623207e4df33f28
2026-03-22 04:39:01.620 DEBUG [src.node.docker_mananger] Container started with ID 1eb271796e57. Setting up logs at ./log/docker/node1_2026-03-22_04-39-01__3c67be1d-c4a3-48d0-9da9-e022f94d1906__wakuorg_nwaku:latest.log
2026-03-22 04:39:01.621 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 60948
2026-03-22 04:39:01.621 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-22 04:39:01.954 ERROR [src.node.docker_mananger] Max retries reached for container d4fe4f085d8c. Exiting log stream.
2026-03-22 04:39:02.621 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60948/health" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:02.625 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-22 04:39:02.625 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-22 04:39:02.625 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60948/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:02.627 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.52.0/tcp/60949/p2p/16Uiu2HAkxo14LUtdQme7C6Yi1bPiHdq3sh5Bc5mQ1JcFSYHdRLPC","/ip4/172.18.52.0/tcp/60950/ws/p2p/16Uiu2HAkxo14LUtdQme7C6Yi1bPiHdq3sh5Bc5mQ1JcFSYHdRLPC"],"enrUri":"enr:-L24QAnhRJLS8InP5jc30DFhb4EggZCw3_HJUsx5iW3JEZ9nCV6W2H7j8jbwH86T9uUOgZz_6s3xh3oLGOsy1V1PSD0CgmlkgnY0gmlwhKwSNACKbXVsdGlhZGRyc5YACASsEjQABu4VAAoErBI0AAbuFt0DgnJzhQADAQAAiXNlY3AyNTZrMaECMgnUZ6r0b3wBWeBvGisnl9cvIzMEbuA7wyBsE-XbOvODdGNwgu4Vg3VkcILuF4V3YWt1MgE"}'
2026-03-22 04:39:02.627 INFO [src.node.waku_node] REST service is ready !!
2026-03-22 04:39:02.628 DEBUG [src.node.waku_node] Starting Node...
2026-03-22 04:39:02.628 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-22 04:39:02.629 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-22 04:39:02.629 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.168.143
2026-03-22 04:39:02.629 DEBUG [src.node.docker_mananger] Generated ports ['60133', '60134', '60135', '60136', '60137']
2026-03-22 04:39:02.630 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-22 04:39:02.630 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-22 04:39:02.630 DEBUG [src.node.waku_node] Using volumes []
2026-03-22 04:39:02.630 DEBUG [src.node.docker_mananger] docker run -i -t -p 60133:60133 -p 60134:60134 -p 60135:60135 -p 60136:60136 -p 60137:60137 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=60135 --rest-port=60133 --tcp-port=60134 --discv5-udp-port=60136 --rest-address=0.0.0.0 --nat=extip:172.18.168.143 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=aebe891af9f42eeb0abd85ca0369d0c4eeb32deea2c75cc4f9d6b21710a14aa4 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=60137 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QAnhRJLS8InP5jc30DFhb4EggZCw3_HJUsx5iW3JEZ9nCV6W2H7j8jbwH86T9uUOgZz_6s3xh3oLGOsy1V1PSD0CgmlkgnY0gmlwhKwSNACKbXVsdGlhZGRyc5YACASsEjQABu4VAAoErBI0AAbuFt0DgnJzhQADAQAAiXNlY3AyNTZrMaECMgnUZ6r0b3wBWeBvGisnl9cvIzMEbuA7wyBsE-XbOvODdGNwgu4Vg3VkcILuF4V3YWt1MgE
2026-03-22 04:39:02.819 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.168.143 waku a3050a2e9e55c954eea60f5a1f1357136806eaaae70df4021d9e0a2525284aa8
2026-03-22 04:39:02.853 DEBUG [src.node.docker_mananger] Container started with ID a3050a2e9e55. Setting up logs at ./log/docker/node2_2026-03-22_04-39-01__3c67be1d-c4a3-48d0-9da9-e022f94d1906__wakuorg_nwaku:latest.log
2026-03-22 04:39:02.853 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 60133
2026-03-22 04:39:02.853 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-22 04:39:03.855 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60133/health" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:03.880 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-22 04:39:03.880 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-22 04:39:03.881 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60133/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:03.886 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.168.143/tcp/60134/p2p/16Uiu2HAmN27fMQV3ybWzxrppHR6C1Str6XVE6JZ1GfYpkG6ijJL6","/ip4/172.18.168.143/tcp/60135/ws/p2p/16Uiu2HAmN27fMQV3ybWzxrppHR6C1Str6XVE6JZ1GfYpkG6ijJL6"],"enrUri":"enr:-L24QPYveoW87cbgkLfHvMoCUuWOrYmV3UvSosEffcmKCJ0lRT_BVFRWd4qrw4rlWxuMSwwYD-MR0CkipAh34238frcCgmlkgnY0gmlwhKwSqI-KbXVsdGlhZGRyc5YACASsEqiPBurmAAoErBKojwbq590DgnJzhQADAQAAiXNlY3AyNTZrMaEDiyKleXCh96n9t7nQM62_2rSlJZFGM1_eE2B4-uxqWDeDdGNwgurmg3VkcILq6IV3YWt1MgE"}'
2026-03-22 04:39:03.888 INFO [src.node.waku_node] REST service is ready !!
2026-03-22 04:39:03.888 DEBUG [src.node.waku_node] Starting Node...
2026-03-22 04:39:03.888 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-22 04:39:03.892 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-22 04:39:03.892 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.80.183
2026-03-22 04:39:03.892 DEBUG [src.node.docker_mananger] Generated ports ['15251', '15252', '15253', '15254', '15255']
2026-03-22 04:39:03.892 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-22 04:39:03.893 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-22 04:39:03.893 DEBUG [src.node.waku_node] Using volumes []
2026-03-22 04:39:03.893 DEBUG [src.node.docker_mananger] docker run -i -t -p 15251:15251 -p 15252:15252 -p 15253:15253 -p 15254:15254 -p 15255:15255 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=15253 --rest-port=15251 --tcp-port=15252 --discv5-udp-port=15254 --rest-address=0.0.0.0 --nat=extip:172.18.80.183 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=2ac4e7f1f8cf43af8dcb65aaa044b089cc0acfa4afea1fd91beb5adcdecaedb4 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=15255 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QAnhRJLS8InP5jc30DFhb4EggZCw3_HJUsx5iW3JEZ9nCV6W2H7j8jbwH86T9uUOgZz_6s3xh3oLGOsy1V1PSD0CgmlkgnY0gmlwhKwSNACKbXVsdGlhZGRyc5YACASsEjQABu4VAAoErBI0AAbuFt0DgnJzhQADAQAAiXNlY3AyNTZrMaECMgnUZ6r0b3wBWeBvGisnl9cvIzMEbuA7wyBsE-XbOvODdGNwgu4Vg3VkcILuF4V3YWt1MgE
2026-03-22 04:39:04.080 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.80.183 waku 3dd7a189d21a99e2daf4f345645b5745e6e6dd44d9573bf046d0c3c90594d034
2026-03-22 04:39:04.115 DEBUG [src.node.docker_mananger] Container started with ID 3dd7a189d21a. Setting up logs at ./log/docker/node3_2026-03-22_04-39-01__3c67be1d-c4a3-48d0-9da9-e022f94d1906__wakuorg_nwaku:latest.log
2026-03-22 04:39:04.116 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 15251
2026-03-22 04:39:04.116 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-22 04:39:05.117 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15251/health" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:05.134 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-22 04:39:05.136 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-22 04:39:05.137 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15251/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:05.142 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.80.183/tcp/15252/p2p/16Uiu2HAm3sHyV2CXxE4tFL33F7kA2ACEC8ZwbHPoW5dQcHUxh6NV","/ip4/172.18.80.183/tcp/15253/ws/p2p/16Uiu2HAm3sHyV2CXxE4tFL33F7kA2ACEC8ZwbHPoW5dQcHUxh6NV"],"enrUri":"enr:-L24QMitWnPxSv8Ly39_PTcK2DZDud6VwPxPo4qj6EKCZsuWTjhyB_2jZ_hatBgdxvF7Dlre218C-ds7WIVJNwGER5ACgmlkgnY0gmlwhKwSULeKbXVsdGlhZGRyc5YACASsElC3BjuUAAoErBJQtwY7ld0DgnJzhQADAQAAiXNlY3AyNTZrMaECfW23Bcq81DqytYn6gFsj71L_tZxK5-H2r_sZb0GAPaKDdGNwgjuUg3VkcII7loV3YWt1MgE"}'
2026-03-22 04:39:05.142 INFO [src.node.waku_node] REST service is ready !!
2026-03-22 04:39:05.142 DEBUG [src.node.waku_node] Starting Node...
2026-03-22 04:39:05.143 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-22 04:39:05.144 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-22 04:39:05.144 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.10.171
2026-03-22 04:39:05.144 DEBUG [src.node.docker_mananger] Generated ports ['50532', '50533', '50534', '50535', '50536']
2026-03-22 04:39:05.144 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-22 04:39:05.145 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-22 04:39:05.145 DEBUG [src.node.waku_node] Using volumes []
2026-03-22 04:39:05.145 DEBUG [src.node.docker_mananger] docker run -i -t -p 50532:50532 -p 50533:50533 -p 50534:50534 -p 50535:50535 -p 50536:50536 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=50534 --rest-port=50532 --tcp-port=50533 --discv5-udp-port=50535 --rest-address=0.0.0.0 --nat=extip:172.18.10.171 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=d71c561cefe45d26a4ba5401c9acb1b561eb97ec6bebba4aeefb8411aab8ddff --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=50536 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QAnhRJLS8InP5jc30DFhb4EggZCw3_HJUsx5iW3JEZ9nCV6W2H7j8jbwH86T9uUOgZz_6s3xh3oLGOsy1V1PSD0CgmlkgnY0gmlwhKwSNACKbXVsdGlhZGRyc5YACASsEjQABu4VAAoErBI0AAbuFt0DgnJzhQADAQAAiXNlY3AyNTZrMaECMgnUZ6r0b3wBWeBvGisnl9cvIzMEbuA7wyBsE-XbOvODdGNwgu4Vg3VkcILuF4V3YWt1MgE
2026-03-22 04:39:05.340 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.10.171 waku 611e5044fc0ef78ace9e4df3dfbd0b97d67f1e6684ab8860977083fadc80f8f8
2026-03-22 04:39:05.375 DEBUG [src.node.docker_mananger] Container started with ID 611e5044fc0e. Setting up logs at ./log/docker/node4_2026-03-22_04-39-01__3c67be1d-c4a3-48d0-9da9-e022f94d1906__wakuorg_nwaku:latest.log
2026-03-22 04:39:05.375 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 50532
2026-03-22 04:39:05.376 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-22 04:39:06.377 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50532/health" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:06.388 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-22 04:39:06.388 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-22 04:39:06.390 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50532/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:06.396 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.10.171/tcp/50533/p2p/16Uiu2HAmUeZtirEcTrnnXFVsiHDZYBgmqkkwSD3Z7Vw7JtPp4Ghs","/ip4/172.18.10.171/tcp/50534/ws/p2p/16Uiu2HAmUeZtirEcTrnnXFVsiHDZYBgmqkkwSD3Z7Vw7JtPp4Ghs"],"enrUri":"enr:-L24QDNht_p0Y2_Z6okJjGgwV57l_55K5oNlfZ3SW-r0Fw43fP-HN1aK3M57KEySIwbnyck76_NLiKwKI-uxurBHTa8CgmlkgnY0gmlwhKwSCquKbXVsdGlhZGRyc5YACASsEgqrBsVlAAoErBIKqwbFZt0DgnJzhQADAQAAiXNlY3AyNTZrMaED7Z9WBF2ALU_apReGy3cuLtECxYbMzNCIeMDPDNWSMcaDdGNwgsVlg3VkcILFZ4V3YWt1MgE"}'
2026-03-22 04:39:06.396 INFO [src.node.waku_node] REST service is ready !!
2026-03-22 04:39:06.397 INFO [tests.e2e.test_network_conditions] Subscribing all nodes to relay topic
2026-03-22 04:39:06.397 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-22 04:39:06.404 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:39:06.405 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60133/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-22 04:39:06.412 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:39:06.413 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15251/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-22 04:39:06.420 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:39:06.420 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:50532/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-22 04:39:06.427 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:39:06.428 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60948/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:06.431 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.80.183/tcp/59884/p2p/16Uiu2HAm3sHyV2CXxE4tFL33F7kA2ACEC8ZwbHPoW5dQcHUxh6NV","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.168.143/tcp/40084/p2p/16Uiu2HAmN27fMQV3ybWzxrppHR6C1Str6XVE6JZ1GfYpkG6ijJL6","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.10.171/tcp/55716/p2p/16Uiu2HAmUeZtirEcTrnnXFVsiHDZYBgmqkkwSD3Z7Vw7JtPp4Ghs","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"UnknownOrigin"}]'
2026-03-22 04:39:06.432 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60133/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:06.436 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.52.0/tcp/60949/p2p/16Uiu2HAkxo14LUtdQme7C6Yi1bPiHdq3sh5Bc5mQ1JcFSYHdRLPC","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"Discv5"}]'
2026-03-22 04:39:06.436 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15251/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:06.438 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.52.0/tcp/60949/p2p/16Uiu2HAkxo14LUtdQme7C6Yi1bPiHdq3sh5Bc5mQ1JcFSYHdRLPC","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"Discv5"}]'
2026-03-22 04:39:06.438 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50532/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:06.440 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.52.0/tcp/60949/p2p/16Uiu2HAkxo14LUtdQme7C6Yi1bPiHdq3sh5Bc5mQ1JcFSYHdRLPC","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"Discv5"}]'
2026-03-22 04:39:06.440 DEBUG [src.libs.common] Sleeping for 30 seconds
2026-03-22 04:39:36.441 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50532/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:39:36.444 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[]'
2026-03-22 04:39:36.444 INFO [tests.e2e.test_network_conditions] Applying 3000ms latency on sender node1
2026-03-22 04:39:36.445 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '6943', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-22 04:39:36.457 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '6943', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-22 04:39:36.467 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '6943', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'delay', '3000ms']
2026-03-22 04:39:36.476 INFO [tests.e2e.test_network_conditions] Sending 50 messages from node1
2026-03-22 04:39:36.476 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:39:42.484 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:39:42.485 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:39:48.491 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:39:48.492 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:39:54.501 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:39:54.502 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:40:00.510 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:40:00.511 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:40:06.519 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:40:06.520 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:40:12.526 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:40:12.526 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:40:18.534 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:40:18.534 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:40:24.540 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:40:24.541 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:40:30.550 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:40:30.550 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:40:36.558 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:40:36.559 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:40:42.565 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:40:42.566 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:40:48.574 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:40:48.574 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:40:54.583 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:40:54.584 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:41:00.593 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:41:00.593 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:41:06.599 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:41:06.600 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:41:12.606 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:41:12.607 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:41:18.615 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:41:18.615 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:41:24.621 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:41:24.622 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:41:30.629 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:41:30.630 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:41:36.636 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:41:36.636 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:41:42.645 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:41:42.646 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:41:48.655 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:41:48.655 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:41:54.661 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:41:54.661 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:42:00.668 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:42:00.668 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:42:06.677 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:42:06.678 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:42:12.684 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:42:12.685 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:42:18.690 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:42:18.691 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:42:24.697 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:42:24.697 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:42:30.703 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:42:30.704 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:42:36.709 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:42:36.710 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:42:42.716 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:42:42.716 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:42:48.725 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:42:48.725 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:42:54.734 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:42:54.734 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:43:00.740 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:43:00.740 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:43:06.749 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:43:06.750 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:43:12.759 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:43:12.759 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:43:18.765 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:43:18.765 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:43:24.773 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:43:24.773 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:43:30.782 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:43:30.783 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:43:36.790 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:43:36.791 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:43:42.799 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:43:42.799 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:43:48.806 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:43:48.807 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:43:54.813 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:43:54.814 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:44:00.821 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:44:00.821 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:44:06.829 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:44:06.829 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:44:12.837 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:44:12.838 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:44:18.846 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:44:18.846 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:44:24.853 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:44:24.854 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:44:30.860 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:44:30.860 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60948/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-22 04:44:36.866 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-22 04:44:36.867 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50532/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-22 04:44:36.872 INFO [src.node.api_clients.base_client] Log line was discarded because it's longer than max_log_line_length=5000
2026-03-22 04:44:36.872 INFO [tests.e2e.test_network_conditions] Node4 received 50 messages (min_expected=25.0, total_sent=50)
2026-03-22 04:44:36.873 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '6943', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-22 04:44:36.885 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-22 04:44:36.885 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-22 04:44:36.886 DEBUG [src.node.waku_node] Stopping container with id 1eb271796e57
2026-03-22 04:44:37.459 DEBUG [src.node.waku_node] Container stopped.
2026-03-22 04:44:37.459 DEBUG [src.node.waku_node] Stopping container with id a3050a2e9e55
2026-03-22 04:44:37.996 DEBUG [src.node.waku_node] Container stopped.
2026-03-22 04:44:37.997 DEBUG [src.node.waku_node] Stopping container with id 3dd7a189d21a
2026-03-22 04:44:38.272 ERROR [src.node.docker_mananger] Max retries reached for container 1eb271796e57. Exiting log stream.
2026-03-22 04:44:38.528 DEBUG [src.node.waku_node] Container stopped.
2026-03-22 04:44:38.529 DEBUG [src.node.waku_node] Stopping container with id 611e5044fc0e
2026-03-22 04:44:38.810 ERROR [src.node.docker_mananger] Max retries reached for container a3050a2e9e55. Exiting log stream.
2026-03-22 04:44:39.073 DEBUG [src.node.waku_node] Container stopped.
2026-03-22 04:44:39.075 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-22 04:44:39.319 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-22 04:44:39.574 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-22 04:44:39.825 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-22 04:44:40.081 DEBUG [src.node.docker_mananger] No errors found in the waku logs.