226 lines
46 KiB
Plaintext

2026-02-01 04:43:27.404 DEBUG [tests.conftest] Running fixture setup: test_id
2026-02-01 04:43:27.405 DEBUG [tests.conftest] Running test: test_relay_7_nodes_3sec_latency with id: 2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88
2026-02-01 04:43:27.406 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-02-01 04:43:27.406 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-02-01 04:43:27.414 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:43:27.414 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:27.422 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:43:27.422 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:27.432 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:43:27.432 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:27.439 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:43:27.440 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:27.448 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:43:27.448 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:27.455 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:43:27.455 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:27.463 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:43:27.463 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:27.470 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:43:27.470 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node6_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:27.477 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:43:27.477 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node7_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:27.477 INFO [tests.e2e.test_network_conditions] Starting nodes with chain bootstrap
2026-02-01 04:43:27.477 DEBUG [src.node.waku_node] Starting Node...
2026-02-01 04:43:27.477 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-01 04:43:27.479 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-01 04:43:27.479 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.222.245
2026-02-01 04:43:27.480 DEBUG [src.node.docker_mananger] Generated ports ['9284', '9285', '9286', '9287', '9288']
2026-02-01 04:43:27.480 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-01 04:43:27.480 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-01 04:43:27.480 DEBUG [src.node.waku_node] Using volumes []
2026-02-01 04:43:27.480 DEBUG [src.node.docker_mananger] docker run -i -t -p 9284:9284 -p 9285:9285 -p 9286:9286 -p 9287:9287 -p 9288:9288 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=9286 --rest-port=9284 --tcp-port=9285 --discv5-udp-port=9287 --rest-address=0.0.0.0 --nat=extip:172.18.222.245 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9b6a8d8aced03db12c19c07ecdeca2eca017f323b6c1abd2043fd8fd8becd8ee --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=9288 --metrics-logging=true --relay=true
2026-02-01 04:43:27.605 ERROR [src.node.docker_mananger] Max retries reached for container 759cedadd712. Exiting log stream.
2026-02-01 04:43:27.674 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.222.245 waku 3ec1c9135b0934f24e43b90efd9a34265ea3b308c6419174dba99b21a6c6e108
2026-02-01 04:43:27.706 DEBUG [src.node.docker_mananger] Container started with ID 3ec1c9135b09. Setting up logs at ./log/docker/node1_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:27.707 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 9284
2026-02-01 04:43:27.707 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-01 04:43:28.309 ERROR [src.node.docker_mananger] Max retries reached for container 380505051d9a. Exiting log stream.
2026-02-01 04:43:28.707 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9284/health" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:28.711 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-01 04:43:28.711 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-01 04:43:28.711 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9284/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:28.713 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.222.245/tcp/9285/p2p/16Uiu2HAmANTC9ofekQXEYWUHUMyQy8FGi5NxdNmfSnPZipLRkrY7","/ip4/172.18.222.245/tcp/9286/ws/p2p/16Uiu2HAmANTC9ofekQXEYWUHUMyQy8FGi5NxdNmfSnPZipLRkrY7"],"enrUri":"enr:-L24QEN5NRAQkhMmKX7QzJemwWjMYUPkfKLAlmIa6TmCWCrMYcnK07Ht_lTRKSmMEUQR4U3HtHMaVQKYvLdx-ri4z_YCgmlkgnY0gmlwhKwS3vWKbXVsdGlhZGRyc5YACASsEt71BiRFAAoErBLe9QYkRt0DgnJzhQADAQAAiXNlY3AyNTZrMaEC3gwab0hwOR4JthPS2xrBrPY9v_EspxyaFWq3FSATkEiDdGNwgiRFg3VkcIIkR4V3YWt1MgE"}'
2026-02-01 04:43:28.714 INFO [src.node.waku_node] REST service is ready !!
2026-02-01 04:43:28.714 DEBUG [src.node.waku_node] Starting Node...
2026-02-01 04:43:28.714 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-01 04:43:28.715 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-01 04:43:28.716 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.124.205
2026-02-01 04:43:28.716 DEBUG [src.node.docker_mananger] Generated ports ['59148', '59149', '59150', '59151', '59152']
2026-02-01 04:43:28.716 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-01 04:43:28.716 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-01 04:43:28.716 DEBUG [src.node.waku_node] Using volumes []
2026-02-01 04:43:28.716 DEBUG [src.node.docker_mananger] docker run -i -t -p 59148:59148 -p 59149:59149 -p 59150:59150 -p 59151:59151 -p 59152:59152 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=59150 --rest-port=59148 --tcp-port=59149 --discv5-udp-port=59151 --rest-address=0.0.0.0 --nat=extip:172.18.124.205 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=de93120e29ac3cdf7c034e5c593bcfaf089f236f0daed8dac6f1950e8becbe10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=59152 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QEN5NRAQkhMmKX7QzJemwWjMYUPkfKLAlmIa6TmCWCrMYcnK07Ht_lTRKSmMEUQR4U3HtHMaVQKYvLdx-ri4z_YCgmlkgnY0gmlwhKwS3vWKbXVsdGlhZGRyc5YACASsEt71BiRFAAoErBLe9QYkRt0DgnJzhQADAQAAiXNlY3AyNTZrMaEC3gwab0hwOR4JthPS2xrBrPY9v_EspxyaFWq3FSATkEiDdGNwgiRFg3VkcIIkR4V3YWt1MgE
2026-02-01 04:43:28.893 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.124.205 waku 83e5bf21dcf7fe227dee84e90a3848def3ccacf18821d1b606cafbff50aa0426
2026-02-01 04:43:28.927 DEBUG [src.node.docker_mananger] Container started with ID 83e5bf21dcf7. Setting up logs at ./log/docker/node2_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:28.928 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 59148
2026-02-01 04:43:28.928 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-01 04:43:29.928 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59148/health" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:29.936 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-01 04:43:29.938 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-01 04:43:29.940 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59148/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:29.944 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.124.205/tcp/59149/p2p/16Uiu2HAmDzsayh9xigT6eWHxGf41otu76yWHqiZeFUtgsJjavG4k","/ip4/172.18.124.205/tcp/59150/ws/p2p/16Uiu2HAmDzsayh9xigT6eWHxGf41otu76yWHqiZeFUtgsJjavG4k"],"enrUri":"enr:-L24QCWZPTcyYfk9K41lYYTwk6BXJQO0LfdQBnPguIwEvzb8J21KIgfBiDPN3iWqK4uuX9oWGwGToDkI8dwYfROjVWgCgmlkgnY0gmlwhKwSfM2KbXVsdGlhZGRyc5YACASsEnzNBucNAAoErBJ8zQbnDt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDE_OnsHZnVK31YFAldIq5yyLQU0geVXLUUXz5A97KY82DdGNwgucNg3VkcILnD4V3YWt1MgE"}'
2026-02-01 04:43:29.944 INFO [src.node.waku_node] REST service is ready !!
2026-02-01 04:43:29.945 DEBUG [src.node.waku_node] Starting Node...
2026-02-01 04:43:29.945 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-01 04:43:29.947 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-01 04:43:29.947 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.146.122
2026-02-01 04:43:29.947 DEBUG [src.node.docker_mananger] Generated ports ['19840', '19841', '19842', '19843', '19844']
2026-02-01 04:43:29.947 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-01 04:43:29.947 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-01 04:43:29.947 DEBUG [src.node.waku_node] Using volumes []
2026-02-01 04:43:29.947 DEBUG [src.node.docker_mananger] docker run -i -t -p 19840:19840 -p 19841:19841 -p 19842:19842 -p 19843:19843 -p 19844:19844 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=19842 --rest-port=19840 --tcp-port=19841 --discv5-udp-port=19843 --rest-address=0.0.0.0 --nat=extip:172.18.146.122 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=818cbb9222cc4e34c9c6f10b47788cd3cb2814ed598c0c572eb8e4950b86c574 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=19844 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QCWZPTcyYfk9K41lYYTwk6BXJQO0LfdQBnPguIwEvzb8J21KIgfBiDPN3iWqK4uuX9oWGwGToDkI8dwYfROjVWgCgmlkgnY0gmlwhKwSfM2KbXVsdGlhZGRyc5YACASsEnzNBucNAAoErBJ8zQbnDt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDE_OnsHZnVK31YFAldIq5yyLQU0geVXLUUXz5A97KY82DdGNwgucNg3VkcILnD4V3YWt1MgE
2026-02-01 04:43:30.125 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.146.122 waku 231e0a5f99aae2e0490a3a6f9ac65bef1459ce24e1b810ab0ac0956e57deefa0
2026-02-01 04:43:30.155 DEBUG [src.node.docker_mananger] Container started with ID 231e0a5f99aa. Setting up logs at ./log/docker/node3_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:30.155 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 19840
2026-02-01 04:43:30.155 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-01 04:43:31.156 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19840/health" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:31.167 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-01 04:43:31.167 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-01 04:43:31.169 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19840/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:31.174 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.146.122/tcp/19841/p2p/16Uiu2HAm9h6PjS8zombLtuhGipwJA19p6EovE7sCFuav4wjRkX3r","/ip4/172.18.146.122/tcp/19842/ws/p2p/16Uiu2HAm9h6PjS8zombLtuhGipwJA19p6EovE7sCFuav4wjRkX3r"],"enrUri":"enr:-L24QLMmA5xKJW1MtFTAngn_coGpNPULqb9X0i1ybCYabCttTXaXwjykwqeL8bp_ucKK6LIjSwj_5Wiq940BxYAWZmACgmlkgnY0gmlwhKwSknqKbXVsdGlhZGRyc5YACASsEpJ6Bk2BAAoErBKSegZNgt0DgnJzhQADAQAAiXNlY3AyNTZrMaEC0_buw_v2KzpqTdpDCslTWkh5Yi1BhQ1O_UIi_tkVJVWDdGNwgk2Bg3VkcIJNg4V3YWt1MgE"}'
2026-02-01 04:43:31.175 INFO [src.node.waku_node] REST service is ready !!
2026-02-01 04:43:31.175 DEBUG [src.node.waku_node] Starting Node...
2026-02-01 04:43:31.175 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-01 04:43:31.178 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-01 04:43:31.178 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.156.251
2026-02-01 04:43:31.178 DEBUG [src.node.docker_mananger] Generated ports ['37343', '37344', '37345', '37346', '37347']
2026-02-01 04:43:31.179 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-01 04:43:31.179 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-01 04:43:31.179 DEBUG [src.node.waku_node] Using volumes []
2026-02-01 04:43:31.179 DEBUG [src.node.docker_mananger] docker run -i -t -p 37343:37343 -p 37344:37344 -p 37345:37345 -p 37346:37346 -p 37347:37347 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=37345 --rest-port=37343 --tcp-port=37344 --discv5-udp-port=37346 --rest-address=0.0.0.0 --nat=extip:172.18.156.251 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=e73ddac8cec5607238a9dc6fd7dfd4ad4d0eafbebf97fcbc9b11fafbdd7c49be --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=37347 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QLMmA5xKJW1MtFTAngn_coGpNPULqb9X0i1ybCYabCttTXaXwjykwqeL8bp_ucKK6LIjSwj_5Wiq940BxYAWZmACgmlkgnY0gmlwhKwSknqKbXVsdGlhZGRyc5YACASsEpJ6Bk2BAAoErBKSegZNgt0DgnJzhQADAQAAiXNlY3AyNTZrMaEC0_buw_v2KzpqTdpDCslTWkh5Yi1BhQ1O_UIi_tkVJVWDdGNwgk2Bg3VkcIJNg4V3YWt1MgE
2026-02-01 04:43:31.359 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.156.251 waku b365fe051dc3fe1595244a19541cdaaf27d7b6a6974dc42eaedfd2ff4dc2b3ba
2026-02-01 04:43:31.391 DEBUG [src.node.docker_mananger] Container started with ID b365fe051dc3. Setting up logs at ./log/docker/node4_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:31.392 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 37343
2026-02-01 04:43:31.392 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-01 04:43:32.393 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:37343/health" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:32.404 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-01 04:43:32.405 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-01 04:43:32.405 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:37343/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:32.410 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.156.251/tcp/37344/p2p/16Uiu2HAmG3tn9SWKMGgTVQD4P57YZemwHwABNvSzfWDa4WMeuGKB","/ip4/172.18.156.251/tcp/37345/ws/p2p/16Uiu2HAmG3tn9SWKMGgTVQD4P57YZemwHwABNvSzfWDa4WMeuGKB"],"enrUri":"enr:-L24QNGnxKLl_tFusQc_JmXK4pBEAu3qgKjSnAtrGxu4OhNXBFUOe2Op2nzJEKYl25x_63WG72V-GeErmFEwHbeVxWsCgmlkgnY0gmlwhKwSnPuKbXVsdGlhZGRyc5YACASsEpz7BpHgAAoErBKc-waR4d0DgnJzhQADAQAAiXNlY3AyNTZrMaEDMnEep0dZQ1Iz_rJ9h6tkgCGRefLjqQKnNDABoRsmDuqDdGNwgpHgg3VkcIKR4oV3YWt1MgE"}'
2026-02-01 04:43:32.411 INFO [src.node.waku_node] REST service is ready !!
2026-02-01 04:43:32.412 DEBUG [src.node.waku_node] Starting Node...
2026-02-01 04:43:32.413 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-01 04:43:32.415 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-01 04:43:32.415 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.239.162
2026-02-01 04:43:32.415 DEBUG [src.node.docker_mananger] Generated ports ['34996', '34997', '34998', '34999', '35000']
2026-02-01 04:43:32.415 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-01 04:43:32.415 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-01 04:43:32.415 DEBUG [src.node.waku_node] Using volumes []
2026-02-01 04:43:32.416 DEBUG [src.node.docker_mananger] docker run -i -t -p 34996:34996 -p 34997:34997 -p 34998:34998 -p 34999:34999 -p 35000:35000 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=34998 --rest-port=34996 --tcp-port=34997 --discv5-udp-port=34999 --rest-address=0.0.0.0 --nat=extip:172.18.239.162 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=edde8aca5b00bbbc49aff52c5aded58dfaefd9d8cc1fb3cce777ce98caff6a07 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=35000 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNGnxKLl_tFusQc_JmXK4pBEAu3qgKjSnAtrGxu4OhNXBFUOe2Op2nzJEKYl25x_63WG72V-GeErmFEwHbeVxWsCgmlkgnY0gmlwhKwSnPuKbXVsdGlhZGRyc5YACASsEpz7BpHgAAoErBKc-waR4d0DgnJzhQADAQAAiXNlY3AyNTZrMaEDMnEep0dZQ1Iz_rJ9h6tkgCGRefLjqQKnNDABoRsmDuqDdGNwgpHgg3VkcIKR4oV3YWt1MgE
2026-02-01 04:43:32.596 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.239.162 waku 172f34ebf6a7bdc3f9e71d521fb5132633e76ecb47d8760409f5cd0cbb1c8ee2
2026-02-01 04:43:32.625 DEBUG [src.node.docker_mananger] Container started with ID 172f34ebf6a7. Setting up logs at ./log/docker/node5_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:32.626 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 34996
2026-02-01 04:43:32.627 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-01 04:43:33.628 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:34996/health" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:33.639 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-01 04:43:33.640 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-01 04:43:33.641 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:34996/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:33.645 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.239.162/tcp/34997/p2p/16Uiu2HAmV7NvjRFqhB6osmpY25C8JqgjepBAjAsDA8ApfeDuThNa","/ip4/172.18.239.162/tcp/34998/ws/p2p/16Uiu2HAmV7NvjRFqhB6osmpY25C8JqgjepBAjAsDA8ApfeDuThNa"],"enrUri":"enr:-L24QFwG7a_JC-5tEbNARNf1CQ5ntYPFzKZcQnoADM_4rf2mGEEVhyZayJTDiHXlxqlvV9R81dhrB5GvgAiLireYytcCgmlkgnY0gmlwhKwS76KKbXVsdGlhZGRyc5YACASsEu-iBoi1AAoErBLvogaItt0DgnJzhQADAQAAiXNlY3AyNTZrMaED9H2eidSSVCZszBuvo4xjQPhtvKrcvuMiLHzoR_WL-JODdGNwgoi1g3VkcIKIt4V3YWt1MgE"}'
2026-02-01 04:43:33.645 INFO [src.node.waku_node] REST service is ready !!
2026-02-01 04:43:33.646 DEBUG [src.node.waku_node] Starting Node...
2026-02-01 04:43:33.647 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-01 04:43:33.649 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-01 04:43:33.649 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.165.235
2026-02-01 04:43:33.649 DEBUG [src.node.docker_mananger] Generated ports ['16566', '16567', '16568', '16569', '16570']
2026-02-01 04:43:33.650 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-01 04:43:33.650 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-01 04:43:33.650 DEBUG [src.node.waku_node] Using volumes []
2026-02-01 04:43:33.650 DEBUG [src.node.docker_mananger] docker run -i -t -p 16566:16566 -p 16567:16567 -p 16568:16568 -p 16569:16569 -p 16570:16570 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=16568 --rest-port=16566 --tcp-port=16567 --discv5-udp-port=16569 --rest-address=0.0.0.0 --nat=extip:172.18.165.235 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=4ae3f8afc56bbc30b7ac5c8dcd34bf9dc8777bf8dd7624fb55256fce6b23a25d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=16570 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QFwG7a_JC-5tEbNARNf1CQ5ntYPFzKZcQnoADM_4rf2mGEEVhyZayJTDiHXlxqlvV9R81dhrB5GvgAiLireYytcCgmlkgnY0gmlwhKwS76KKbXVsdGlhZGRyc5YACASsEu-iBoi1AAoErBLvogaItt0DgnJzhQADAQAAiXNlY3AyNTZrMaED9H2eidSSVCZszBuvo4xjQPhtvKrcvuMiLHzoR_WL-JODdGNwgoi1g3VkcIKIt4V3YWt1MgE
2026-02-01 04:43:33.843 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.165.235 waku 6588d76984a004c645a5e858d366697f4cfacd980b54036fe5d94b6764aabca4
2026-02-01 04:43:33.876 DEBUG [src.node.docker_mananger] Container started with ID 6588d76984a0. Setting up logs at ./log/docker/node6_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:33.877 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 16566
2026-02-01 04:43:33.877 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-01 04:43:34.877 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16566/health" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:34.886 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-01 04:43:34.888 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-01 04:43:34.888 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16566/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:34.894 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.165.235/tcp/16567/p2p/16Uiu2HAmS816krtcnjMvX5DwoQC4JsmDqNarkhCodPBy4nn2BfSB","/ip4/172.18.165.235/tcp/16568/ws/p2p/16Uiu2HAmS816krtcnjMvX5DwoQC4JsmDqNarkhCodPBy4nn2BfSB"],"enrUri":"enr:-L24QAPc8xtr2F9-f0cFoqw52JCBWDGi4dy83qHe-xuMnv8TR0iYL1SpnnG9R42YbY6vZEDZ59EgFICQbcXyILqrWm4CgmlkgnY0gmlwhKwSpeuKbXVsdGlhZGRyc5YACASsEqXrBkC3AAoErBKl6wZAuN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDyBN1kwpT3vzwrv3V1xg5VtFyjrEx4r5nQMOHEaPEhYyDdGNwgkC3g3VkcIJAuYV3YWt1MgE"}'
2026-02-01 04:43:34.894 INFO [src.node.waku_node] REST service is ready !!
2026-02-01 04:43:34.894 DEBUG [src.node.waku_node] Starting Node...
2026-02-01 04:43:34.895 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-01 04:43:34.897 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-01 04:43:34.897 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.5.152
2026-02-01 04:43:34.897 DEBUG [src.node.docker_mananger] Generated ports ['24167', '24168', '24169', '24170', '24171']
2026-02-01 04:43:34.897 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-01 04:43:34.897 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-01 04:43:34.897 DEBUG [src.node.waku_node] Using volumes []
2026-02-01 04:43:34.897 DEBUG [src.node.docker_mananger] docker run -i -t -p 24167:24167 -p 24168:24168 -p 24169:24169 -p 24170:24170 -p 24171:24171 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=24169 --rest-port=24167 --tcp-port=24168 --discv5-udp-port=24170 --rest-address=0.0.0.0 --nat=extip:172.18.5.152 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9fab35ade0f0fed6290ebcc7cfbda40a59b382f3cf9b2abed13979af277bc06f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=24171 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QAPc8xtr2F9-f0cFoqw52JCBWDGi4dy83qHe-xuMnv8TR0iYL1SpnnG9R42YbY6vZEDZ59EgFICQbcXyILqrWm4CgmlkgnY0gmlwhKwSpeuKbXVsdGlhZGRyc5YACASsEqXrBkC3AAoErBKl6wZAuN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDyBN1kwpT3vzwrv3V1xg5VtFyjrEx4r5nQMOHEaPEhYyDdGNwgkC3g3VkcIJAuYV3YWt1MgE
2026-02-01 04:43:35.098 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.5.152 waku 60bdf86ead7f3992b60a3fbfd48c0264889b909667c1374356c5d582c3f7ecdc
2026-02-01 04:43:35.129 DEBUG [src.node.docker_mananger] Container started with ID 60bdf86ead7f. Setting up logs at ./log/docker/node7_2026-02-01_04-43-27__0500ca9d-8d09-49ff-970b-3ba7f828fb88__wakuorg_nwaku:latest.log
2026-02-01 04:43:35.129 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 24167
2026-02-01 04:43:35.130 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-01 04:43:36.131 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24167/health" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:36.139 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"READY"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-01 04:43:36.140 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-01 04:43:36.140 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24167/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:36.145 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.5.152/tcp/24168/p2p/16Uiu2HAmFpnSiHcY7LoaUnwTeaERL9sZcfP3ULYxB9ZRZMshkSkJ","/ip4/172.18.5.152/tcp/24169/ws/p2p/16Uiu2HAmFpnSiHcY7LoaUnwTeaERL9sZcfP3ULYxB9ZRZMshkSkJ"],"enrUri":"enr:-L24QCt1ew_08KqacszYKru1gZov8PsbV8zcgO-yHZPbXEcyHITgDrLdeBCcDBzj1f4I0gKJ_0ptd0yAfae9icf8aH8CgmlkgnY0gmlwhKwSBZiKbXVsdGlhZGRyc5YACASsEgWYBl5oAAoErBIFmAZead0DgnJzhQADAQAAiXNlY3AyNTZrMaEDLxVn-PwdZHavslfsn6DU-MmhjqXOZ0wTpUKLZPFXP8uDdGNwgl5og3VkcIJeaoV3YWt1MgE"}'
2026-02-01 04:43:36.146 INFO [src.node.waku_node] REST service is ready !!
2026-02-01 04:43:36.146 INFO [tests.e2e.test_network_conditions] Subscribing all nodes to relay topic
2026-02-01 04:43:36.146 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:9284/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-02-01 04:43:36.150 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:43:36.151 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:59148/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-02-01 04:43:36.157 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:43:36.157 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19840/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-02-01 04:43:36.163 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:43:36.165 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:37343/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-02-01 04:43:36.171 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:43:36.172 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:34996/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-02-01 04:43:36.180 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:43:36.183 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16566/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-02-01 04:43:36.190 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:43:36.190 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:24167/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-02-01 04:43:36.196 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:43:36.197 INFO [tests.e2e.test_network_conditions] Waiting for autoconnection
2026-02-01 04:43:36.197 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9284/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:36.201 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.124.205/tcp/35130/p2p/16Uiu2HAmDzsayh9xigT6eWHxGf41otu76yWHqiZeFUtgsJjavG4k","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-127-ga4e44d","origin":"Discv5"}]'
2026-02-01 04:43:36.201 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59148/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:36.204 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.146.122/tcp/42636/p2p/16Uiu2HAm9h6PjS8zombLtuhGipwJA19p6EovE7sCFuav4wjRkX3r","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-127-ga4e44d","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.222.245/tcp/9285/p2p/16Uiu2HAmANTC9ofekQXEYWUHUMyQy8FGi5NxdNmfSnPZipLRkrY7","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-127-ga4e44d","origin":"Discv5"}]'
2026-02-01 04:43:36.204 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19840/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:36.206 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.124.205/tcp/59149/p2p/16Uiu2HAmDzsayh9xigT6eWHxGf41otu76yWHqiZeFUtgsJjavG4k","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-127-ga4e44d","origin":"Discv5"},{"multiaddr":"/ip4/172.18.156.251/tcp/49620/p2p/16Uiu2HAmG3tn9SWKMGgTVQD4P57YZemwHwABNvSzfWDa4WMeuGKB","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-127-ga4e44d","origin":"UnknownOrigin"}]'
2026-02-01 04:43:36.207 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:37343/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:36.209 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.239.162/tcp/47346/p2p/16Uiu2HAmV7NvjRFqhB6osmpY25C8JqgjepBAjAsDA8ApfeDuThNa","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-127-ga4e44d","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.146.122/tcp/19841/p2p/16Uiu2HAm9h6PjS8zombLtuhGipwJA19p6EovE7sCFuav4wjRkX3r","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-127-ga4e44d","origin":"Discv5"}]'
2026-02-01 04:43:36.209 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:34996/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:36.211 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.165.235/tcp/35966/p2p/16Uiu2HAmS816krtcnjMvX5DwoQC4JsmDqNarkhCodPBy4nn2BfSB","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-127-ga4e44d","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.156.251/tcp/37344/p2p/16Uiu2HAmG3tn9SWKMGgTVQD4P57YZemwHwABNvSzfWDa4WMeuGKB","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-127-ga4e44d","origin":"Discv5"}]'
2026-02-01 04:43:36.212 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16566/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:36.214 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.239.162/tcp/34997/p2p/16Uiu2HAmV7NvjRFqhB6osmpY25C8JqgjepBAjAsDA8ApfeDuThNa","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-127-ga4e44d","origin":"Discv5"},{"multiaddr":"/ip4/172.18.5.152/tcp/59990/p2p/16Uiu2HAmFpnSiHcY7LoaUnwTeaERL9sZcfP3ULYxB9ZRZMshkSkJ","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-127-ga4e44d","origin":"UnknownOrigin"}]'
2026-02-01 04:43:36.214 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24167/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:43:36.216 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.165.235/tcp/16567/p2p/16Uiu2HAmS816krtcnjMvX5DwoQC4JsmDqNarkhCodPBy4nn2BfSB","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-127-ga4e44d","origin":"Discv5"}]'
2026-02-01 04:43:36.216 DEBUG [src.libs.common] Sleeping for 60 seconds
2026-02-01 04:44:36.217 INFO [tests.e2e.test_network_conditions] Applying 3s latency to node3
2026-02-01 04:44:36.218 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '5350', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-02-01 04:44:36.249 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '5350', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'delay', '3000ms']
2026-02-01 04:44:36.280 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19840/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:44:42.284 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[]'
2026-02-01 04:44:42.284 INFO [tests.e2e.test_network_conditions] Observed GET latency on node3: 6.00s
2026-02-01 04:44:42.286 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-02-01 04:44:42.287 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-02-01 04:44:42.287 DEBUG [src.node.waku_node] Stopping container with id 3ec1c9135b09
2026-02-01 04:44:42.894 DEBUG [src.node.waku_node] Container stopped.
2026-02-01 04:44:42.894 DEBUG [src.node.waku_node] Stopping container with id 83e5bf21dcf7
2026-02-01 04:44:43.445 DEBUG [src.node.waku_node] Container stopped.
2026-02-01 04:44:43.445 DEBUG [src.node.waku_node] Stopping container with id 231e0a5f99aa
2026-02-01 04:44:43.708 ERROR [src.node.docker_mananger] Max retries reached for container 3ec1c9135b09. Exiting log stream.
2026-02-01 04:44:44.030 DEBUG [src.node.waku_node] Container stopped.
2026-02-01 04:44:44.030 DEBUG [src.node.waku_node] Stopping container with id b365fe051dc3
2026-02-01 04:44:44.267 ERROR [src.node.docker_mananger] Max retries reached for container 83e5bf21dcf7. Exiting log stream.
2026-02-01 04:44:44.580 DEBUG [src.node.waku_node] Container stopped.
2026-02-01 04:44:44.581 DEBUG [src.node.waku_node] Stopping container with id 172f34ebf6a7
2026-02-01 04:44:44.845 ERROR [src.node.docker_mananger] Max retries reached for container 231e0a5f99aa. Exiting log stream.
2026-02-01 04:44:45.154 DEBUG [src.node.waku_node] Container stopped.
2026-02-01 04:44:45.154 DEBUG [src.node.waku_node] Stopping container with id 6588d76984a0
2026-02-01 04:44:45.392 ERROR [src.node.docker_mananger] Max retries reached for container b365fe051dc3. Exiting log stream.
2026-02-01 04:44:45.968 ERROR [src.node.docker_mananger] Max retries reached for container 172f34ebf6a7. Exiting log stream.
2026-02-01 04:44:55.733 DEBUG [src.node.waku_node] Container stopped.
2026-02-01 04:44:55.734 DEBUG [src.node.waku_node] Stopping container with id 60bdf86ead7f
2026-02-01 04:44:56.276 DEBUG [src.node.waku_node] Container stopped.
2026-02-01 04:44:56.276 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-02-01 04:44:56.358 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-01 04:44:56.439 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-01 04:44:56.516 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-01 04:44:56.603 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-01 04:44:56.684 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-01 04:44:56.700 ERROR [src.node.docker_mananger] Max retries reached for container 6588d76984a0. Exiting log stream.
2026-02-01 04:44:56.759 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-01 04:44:56.838 DEBUG [src.node.docker_mananger] No errors found in the waku logs.