226 lines
47 KiB
Plaintext

2026-03-25 04:37:54.975 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-25 04:37:54.975 DEBUG [tests.conftest] Running test: test_relay_7_nodes_3sec_latency with id: 2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd
2026-03-25 04:37:54.976 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-25 04:37:54.976 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-25 04:37:54.983 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-25 04:37:54.983 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:54.989 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-25 04:37:54.989 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:54.995 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-25 04:37:54.996 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:55.002 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-25 04:37:55.002 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:55.008 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-25 04:37:55.008 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:55.014 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-25 04:37:55.014 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:55.020 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-25 04:37:55.020 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:55.026 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-25 04:37:55.026 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node6_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:55.032 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-25 04:37:55.032 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node7_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:55.032 INFO [tests.e2e.test_network_conditions] Starting nodes with chain bootstrap
2026-03-25 04:37:55.032 DEBUG [src.node.waku_node] Starting Node...
2026-03-25 04:37:55.032 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-25 04:37:55.034 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-25 04:37:55.034 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.44.98
2026-03-25 04:37:55.034 DEBUG [src.node.docker_mananger] Generated ports ['24687', '24688', '24689', '24690', '24691']
2026-03-25 04:37:55.034 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-25 04:37:55.034 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-25 04:37:55.035 DEBUG [src.node.waku_node] Using volumes []
2026-03-25 04:37:55.035 DEBUG [src.node.docker_mananger] docker run -i -t -p 24687:24687 -p 24688:24688 -p 24689:24689 -p 24690:24690 -p 24691:24691 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=24689 --rest-port=24687 --tcp-port=24688 --discv5-udp-port=24690 --rest-address=0.0.0.0 --nat=extip:172.18.44.98 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=0ffd961ccddcebff21bfc1ab160c957ce2b67fccc7bb2fd32d6a1ccb52dcb7b8 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=24691 --metrics-logging=true --relay=true
2026-03-25 04:37:55.175 ERROR [src.node.docker_mananger] Max retries reached for container a7d78db12941. Exiting log stream.
2026-03-25 04:37:55.237 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.44.98 waku b4d53847e13f7ddf7fe2ef8b67718cb55f363505b9552c667b142f423aedee06
2026-03-25 04:37:55.274 DEBUG [src.node.docker_mananger] Container started with ID b4d53847e13f. Setting up logs at ./log/docker/node1_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:55.274 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 24687
2026-03-25 04:37:55.275 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-25 04:37:55.734 ERROR [src.node.docker_mananger] Max retries reached for container 45c8b02f7e73. Exiting log stream.
2026-03-25 04:37:56.276 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24687/health" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:37:56.279 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-25 04:37:56.279 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-25 04:37:56.279 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24687/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:37:56.282 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.44.98/tcp/24688/p2p/16Uiu2HAmLYWdkLehMMU7pKPPA1bz2SBautR2rVagAzj45oALk82m","/ip4/172.18.44.98/tcp/24689/ws/p2p/16Uiu2HAmLYWdkLehMMU7pKPPA1bz2SBautR2rVagAzj45oALk82m"],"enrUri":"enr:-L24QJNScTtxOrKe8-EYQwc8wrET__v7xZNR0X8TYMN4UloxQb6Jax2ao6GPFnLqRuNRwjlm0emYoXbYup2PprJdfR8CgmlkgnY0gmlwhKwSLGKKbXVsdGlhZGRyc5YACASsEixiBmBwAAoErBIsYgZgcd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDdTSr9OO6OIw4GgHFtiD0NYvVxtyzBb8hUjgOAnwTCGqDdGNwgmBwg3VkcIJgcoV3YWt1MgE"}'
2026-03-25 04:37:56.282 INFO [src.node.waku_node] REST service is ready !!
2026-03-25 04:37:56.282 DEBUG [src.node.waku_node] Starting Node...
2026-03-25 04:37:56.282 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-25 04:37:56.284 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-25 04:37:56.284 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.224.238
2026-03-25 04:37:56.284 DEBUG [src.node.docker_mananger] Generated ports ['27795', '27796', '27797', '27798', '27799']
2026-03-25 04:37:56.284 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-25 04:37:56.285 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-25 04:37:56.285 DEBUG [src.node.waku_node] Using volumes []
2026-03-25 04:37:56.285 DEBUG [src.node.docker_mananger] docker run -i -t -p 27795:27795 -p 27796:27796 -p 27797:27797 -p 27798:27798 -p 27799:27799 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=27797 --rest-port=27795 --tcp-port=27796 --discv5-udp-port=27798 --rest-address=0.0.0.0 --nat=extip:172.18.224.238 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=2ce1b9ea5208425d01893cce7b9d445581ebccceba1080bddd3fefaa11f1a37d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=27799 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QJNScTtxOrKe8-EYQwc8wrET__v7xZNR0X8TYMN4UloxQb6Jax2ao6GPFnLqRuNRwjlm0emYoXbYup2PprJdfR8CgmlkgnY0gmlwhKwSLGKKbXVsdGlhZGRyc5YACASsEixiBmBwAAoErBIsYgZgcd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDdTSr9OO6OIw4GgHFtiD0NYvVxtyzBb8hUjgOAnwTCGqDdGNwgmBwg3VkcIJgcoV3YWt1MgE
2026-03-25 04:37:56.472 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.224.238 waku c9abaa387fcc9a720c96fddd06c19c917e5d636170f1fc2caaa5f928c631a6dd
2026-03-25 04:37:56.507 DEBUG [src.node.docker_mananger] Container started with ID c9abaa387fcc. Setting up logs at ./log/docker/node2_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:56.508 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 27795
2026-03-25 04:37:56.508 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-25 04:37:57.510 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27795/health" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:37:57.532 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-25 04:37:57.534 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-25 04:37:57.535 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27795/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:37:57.541 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.224.238/tcp/27796/p2p/16Uiu2HAmRDGAeE14SYfhGiTJW91tcrNg41wRcwR3sCEhB8rvAX7E","/ip4/172.18.224.238/tcp/27797/ws/p2p/16Uiu2HAmRDGAeE14SYfhGiTJW91tcrNg41wRcwR3sCEhB8rvAX7E"],"enrUri":"enr:-L24QETspP7Vi0LDdd_D0_71M7tnJ9B4t3BjCYL1ooXB_eKSSxiOz8TTw0K4nwdSBwrojV9AiwaX0TaAyEB9_-lbRcoCgmlkgnY0gmlwhKwS4O6KbXVsdGlhZGRyc5YACASsEuDuBmyUAAoErBLg7gZsld0DgnJzhQADAQAAiXNlY3AyNTZrMaEDupC2c1CzJnB3HZa_GJCUNFGxrUk69ZkoBpGnQEWSebmDdGNwgmyUg3VkcIJsloV3YWt1MgE"}'
2026-03-25 04:37:57.542 INFO [src.node.waku_node] REST service is ready !!
2026-03-25 04:37:57.544 DEBUG [src.node.waku_node] Starting Node...
2026-03-25 04:37:57.544 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-25 04:37:57.548 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-25 04:37:57.549 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.191.6
2026-03-25 04:37:57.549 DEBUG [src.node.docker_mananger] Generated ports ['9104', '9105', '9106', '9107', '9108']
2026-03-25 04:37:57.550 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-25 04:37:57.550 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-25 04:37:57.550 DEBUG [src.node.waku_node] Using volumes []
2026-03-25 04:37:57.550 DEBUG [src.node.docker_mananger] docker run -i -t -p 9104:9104 -p 9105:9105 -p 9106:9106 -p 9107:9107 -p 9108:9108 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=9106 --rest-port=9104 --tcp-port=9105 --discv5-udp-port=9107 --rest-address=0.0.0.0 --nat=extip:172.18.191.6 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=531ce186d86fa8e8d3c58ce7e3ad4fbe64e537b7052acadf0cf66a9addcbf1ec --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=9108 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QETspP7Vi0LDdd_D0_71M7tnJ9B4t3BjCYL1ooXB_eKSSxiOz8TTw0K4nwdSBwrojV9AiwaX0TaAyEB9_-lbRcoCgmlkgnY0gmlwhKwS4O6KbXVsdGlhZGRyc5YACASsEuDuBmyUAAoErBLg7gZsld0DgnJzhQADAQAAiXNlY3AyNTZrMaEDupC2c1CzJnB3HZa_GJCUNFGxrUk69ZkoBpGnQEWSebmDdGNwgmyUg3VkcIJsloV3YWt1MgE
2026-03-25 04:37:57.746 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.191.6 waku 7316a32f416dd12cb80bb7b9e00f50147673ed925afffc3cb9dbc9fc77cba5f4
2026-03-25 04:37:57.783 DEBUG [src.node.docker_mananger] Container started with ID 7316a32f416d. Setting up logs at ./log/docker/node3_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:57.784 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 9104
2026-03-25 04:37:57.784 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-25 04:37:58.784 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9104/health" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:37:58.798 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-25 04:37:58.799 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-25 04:37:58.799 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9104/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:37:58.808 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.191.6/tcp/9105/p2p/16Uiu2HAkxzqMvYZD9sDn9UkUX8CGkChU8n3paYwT4TGL8SZ5KXqv","/ip4/172.18.191.6/tcp/9106/ws/p2p/16Uiu2HAkxzqMvYZD9sDn9UkUX8CGkChU8n3paYwT4TGL8SZ5KXqv"],"enrUri":"enr:-L24QHWzdI3oZQdjjypWJHCq-BeJb520Wk7MSYEPrGMKw6xEK8sfHk04pKPJhUPGjrOj-mJ84ZC8gTUR-1tVLOz9Dn8CgmlkgnY0gmlwhKwSvwaKbXVsdGlhZGRyc5YACASsEr8GBiORAAoErBK_BgYjkt0DgnJzhQADAQAAiXNlY3AyNTZrMaECNRHV0m0ru-RhoOBeK98TyuaiMS_6m_Lz05CX-Rt20x2DdGNwgiORg3VkcIIjk4V3YWt1MgE"}'
2026-03-25 04:37:58.808 INFO [src.node.waku_node] REST service is ready !!
2026-03-25 04:37:58.809 DEBUG [src.node.waku_node] Starting Node...
2026-03-25 04:37:58.809 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-25 04:37:58.810 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-25 04:37:58.810 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.177.170
2026-03-25 04:37:58.810 DEBUG [src.node.docker_mananger] Generated ports ['16599', '16600', '16601', '16602', '16603']
2026-03-25 04:37:58.811 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-25 04:37:58.811 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-25 04:37:58.811 DEBUG [src.node.waku_node] Using volumes []
2026-03-25 04:37:58.811 DEBUG [src.node.docker_mananger] docker run -i -t -p 16599:16599 -p 16600:16600 -p 16601:16601 -p 16602:16602 -p 16603:16603 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=16601 --rest-port=16599 --tcp-port=16600 --discv5-udp-port=16602 --rest-address=0.0.0.0 --nat=extip:172.18.177.170 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=2aafc1eeccfdf0db769163e37ce9ce39a9daf477b1fcdf1cccdef9fc6590eae0 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=16603 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QHWzdI3oZQdjjypWJHCq-BeJb520Wk7MSYEPrGMKw6xEK8sfHk04pKPJhUPGjrOj-mJ84ZC8gTUR-1tVLOz9Dn8CgmlkgnY0gmlwhKwSvwaKbXVsdGlhZGRyc5YACASsEr8GBiORAAoErBK_BgYjkt0DgnJzhQADAQAAiXNlY3AyNTZrMaECNRHV0m0ru-RhoOBeK98TyuaiMS_6m_Lz05CX-Rt20x2DdGNwgiORg3VkcIIjk4V3YWt1MgE
2026-03-25 04:37:59.047 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.177.170 waku 5c691cd1231623ad2f7c25a64dfeb4d3e17cc84a50da1a0f1c087b9f0b1e19e7
2026-03-25 04:37:59.081 DEBUG [src.node.docker_mananger] Container started with ID 5c691cd12316. Setting up logs at ./log/docker/node4_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:37:59.082 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 16599
2026-03-25 04:37:59.082 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-25 04:38:00.082 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16599/health" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:00.102 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-25 04:38:00.103 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-25 04:38:00.104 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16599/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:00.109 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.177.170/tcp/16600/p2p/16Uiu2HAmDDnRdyBGv7nxYuDzuU44deWLPKYZfbv9DscS6c9U17Rx","/ip4/172.18.177.170/tcp/16601/ws/p2p/16Uiu2HAmDDnRdyBGv7nxYuDzuU44deWLPKYZfbv9DscS6c9U17Rx"],"enrUri":"enr:-L24QABDA6AH3gxTNFMb3PI3t6Ya16FZnCpgcnqHIIwPrKN3St8SfJ0uFWOQYwfQC9vKoCYHNFiYeTIliyqCSlNW6NUCgmlkgnY0gmlwhKwSsaqKbXVsdGlhZGRyc5YACASsErGqBkDYAAoErBKxqgZA2d0DgnJzhQADAQAAiXNlY3AyNTZrMaEDCGau2tHZcLkhfhA0QCw_0xasiERcR62J63l0a7ytQG-DdGNwgkDYg3VkcIJA2oV3YWt1MgE"}'
2026-03-25 04:38:00.109 INFO [src.node.waku_node] REST service is ready !!
2026-03-25 04:38:00.109 DEBUG [src.node.waku_node] Starting Node...
2026-03-25 04:38:00.111 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-25 04:38:00.113 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-25 04:38:00.113 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.203.235
2026-03-25 04:38:00.113 DEBUG [src.node.docker_mananger] Generated ports ['19484', '19485', '19486', '19487', '19488']
2026-03-25 04:38:00.114 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-25 04:38:00.114 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-25 04:38:00.114 DEBUG [src.node.waku_node] Using volumes []
2026-03-25 04:38:00.114 DEBUG [src.node.docker_mananger] docker run -i -t -p 19484:19484 -p 19485:19485 -p 19486:19486 -p 19487:19487 -p 19488:19488 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=19486 --rest-port=19484 --tcp-port=19485 --discv5-udp-port=19487 --rest-address=0.0.0.0 --nat=extip:172.18.203.235 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=2babdd1d4a8deeb37afe1ffe7d6406e4ca6cefeffc7adddf06fa8fa0a3ed14c7 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=19488 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QABDA6AH3gxTNFMb3PI3t6Ya16FZnCpgcnqHIIwPrKN3St8SfJ0uFWOQYwfQC9vKoCYHNFiYeTIliyqCSlNW6NUCgmlkgnY0gmlwhKwSsaqKbXVsdGlhZGRyc5YACASsErGqBkDYAAoErBKxqgZA2d0DgnJzhQADAQAAiXNlY3AyNTZrMaEDCGau2tHZcLkhfhA0QCw_0xasiERcR62J63l0a7ytQG-DdGNwgkDYg3VkcIJA2oV3YWt1MgE
2026-03-25 04:38:00.328 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.203.235 waku 13c177992cda98b8ea8c5161ba47da45b83ab6c9446aa0a92cdb816940df6a0d
2026-03-25 04:38:00.363 DEBUG [src.node.docker_mananger] Container started with ID 13c177992cda. Setting up logs at ./log/docker/node5_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:38:00.363 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 19484
2026-03-25 04:38:00.364 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-25 04:38:01.365 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19484/health" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:01.377 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-25 04:38:01.378 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-25 04:38:01.378 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19484/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:01.384 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.203.235/tcp/19485/p2p/16Uiu2HAmAcgqY5a8PvqGgSkyg74hc86xcZkRMoEWcdjJYte1yd8w","/ip4/172.18.203.235/tcp/19486/ws/p2p/16Uiu2HAmAcgqY5a8PvqGgSkyg74hc86xcZkRMoEWcdjJYte1yd8w"],"enrUri":"enr:-L24QJEo_2RNePq1tWaxGgFaK8u_95LgnzHb-AuqUKo6ADjIB8HbUJfGaqez9CRovxqNTSAxiLgzngq_HiTEKppVIa0CgmlkgnY0gmlwhKwSy-uKbXVsdGlhZGRyc5YACASsEsvrBkwdAAoErBLL6wZMHt0DgnJzhQADAQAAiXNlY3AyNTZrMaEC4bGpm7UmMcrrfwSj5p3qV0EUj_0JTIW-p_p2FT2ZRPyDdGNwgkwdg3VkcIJMH4V3YWt1MgE"}'
2026-03-25 04:38:01.385 INFO [src.node.waku_node] REST service is ready !!
2026-03-25 04:38:01.387 DEBUG [src.node.waku_node] Starting Node...
2026-03-25 04:38:01.387 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-25 04:38:01.391 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-25 04:38:01.392 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.9.54
2026-03-25 04:38:01.392 DEBUG [src.node.docker_mananger] Generated ports ['53302', '53303', '53304', '53305', '53306']
2026-03-25 04:38:01.392 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-25 04:38:01.392 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-25 04:38:01.392 DEBUG [src.node.waku_node] Using volumes []
2026-03-25 04:38:01.392 DEBUG [src.node.docker_mananger] docker run -i -t -p 53302:53302 -p 53303:53303 -p 53304:53304 -p 53305:53305 -p 53306:53306 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=53304 --rest-port=53302 --tcp-port=53303 --discv5-udp-port=53305 --rest-address=0.0.0.0 --nat=extip:172.18.9.54 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=30fe1db5ad27e1cb41c1657fb86c54ceabfbd842d1a601133b92eacdb1e9c758 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=53306 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QJEo_2RNePq1tWaxGgFaK8u_95LgnzHb-AuqUKo6ADjIB8HbUJfGaqez9CRovxqNTSAxiLgzngq_HiTEKppVIa0CgmlkgnY0gmlwhKwSy-uKbXVsdGlhZGRyc5YACASsEsvrBkwdAAoErBLL6wZMHt0DgnJzhQADAQAAiXNlY3AyNTZrMaEC4bGpm7UmMcrrfwSj5p3qV0EUj_0JTIW-p_p2FT2ZRPyDdGNwgkwdg3VkcIJMH4V3YWt1MgE
2026-03-25 04:38:01.600 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.9.54 waku e6fb9a96abfc01acbb53aa649c1783e4c9230fee098289cbecaa3038931df5fa
2026-03-25 04:38:01.635 DEBUG [src.node.docker_mananger] Container started with ID e6fb9a96abfc. Setting up logs at ./log/docker/node6_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:38:01.635 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 53302
2026-03-25 04:38:01.636 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-25 04:38:02.637 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53302/health" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:02.654 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-25 04:38:02.658 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-25 04:38:02.658 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53302/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:02.664 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.9.54/tcp/53303/p2p/16Uiu2HAm5DoJUpeeVW8TEH9Wc7Myv865pzk25feVr4hAQG2f2ZyW","/ip4/172.18.9.54/tcp/53304/ws/p2p/16Uiu2HAm5DoJUpeeVW8TEH9Wc7Myv865pzk25feVr4hAQG2f2ZyW"],"enrUri":"enr:-L24QJSeBHyY2g1XuqFVBDPTH5pIsv7gdSh7hwsV_uIhIsemVGJlHnGhU9OK3lRD5g2YwP9tYHvTIbhbbN90b52glMsCgmlkgnY0gmlwhKwSCTaKbXVsdGlhZGRyc5YACASsEgk2BtA3AAoErBIJNgbQON0DgnJzhQADAQAAiXNlY3AyNTZrMaECkYotxI7qLXsChm1icWPMhdkQOmpWTsPrOtsSQ3QZzbWDdGNwgtA3g3VkcILQOYV3YWt1MgE"}'
2026-03-25 04:38:02.664 INFO [src.node.waku_node] REST service is ready !!
2026-03-25 04:38:02.664 DEBUG [src.node.waku_node] Starting Node...
2026-03-25 04:38:02.665 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-25 04:38:02.667 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-25 04:38:02.667 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.89.3
2026-03-25 04:38:02.667 DEBUG [src.node.docker_mananger] Generated ports ['53733', '53734', '53735', '53736', '53737']
2026-03-25 04:38:02.668 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-25 04:38:02.668 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-25 04:38:02.668 DEBUG [src.node.waku_node] Using volumes []
2026-03-25 04:38:02.668 DEBUG [src.node.docker_mananger] docker run -i -t -p 53733:53733 -p 53734:53734 -p 53735:53735 -p 53736:53736 -p 53737:53737 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=53735 --rest-port=53733 --tcp-port=53734 --discv5-udp-port=53736 --rest-address=0.0.0.0 --nat=extip:172.18.89.3 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=dfa4d8db8ac4a45f97b15f5cdccbcdec9b25d8256acac4fa9cdb11ebeecc82e2 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=53737 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QJSeBHyY2g1XuqFVBDPTH5pIsv7gdSh7hwsV_uIhIsemVGJlHnGhU9OK3lRD5g2YwP9tYHvTIbhbbN90b52glMsCgmlkgnY0gmlwhKwSCTaKbXVsdGlhZGRyc5YACASsEgk2BtA3AAoErBIJNgbQON0DgnJzhQADAQAAiXNlY3AyNTZrMaECkYotxI7qLXsChm1icWPMhdkQOmpWTsPrOtsSQ3QZzbWDdGNwgtA3g3VkcILQOYV3YWt1MgE
2026-03-25 04:38:02.875 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.89.3 waku 60191545f466735e0a11427b36394e6de780ae034a083b889a2a716053630205
2026-03-25 04:38:02.908 DEBUG [src.node.docker_mananger] Container started with ID 60191545f466. Setting up logs at ./log/docker/node7_2026-03-25_04-37-54__09a2ccbd-4e05-4b15-b137-81b1e14d45fd__wakuorg_nwaku:latest.log
2026-03-25 04:38:02.909 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 53733
2026-03-25 04:38:02.909 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-25 04:38:03.910 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53733/health" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:03.927 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-25 04:38:03.929 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-25 04:38:03.930 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53733/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:03.937 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.89.3/tcp/53734/p2p/16Uiu2HAkupbkd82QosVzfeYQTuNy7CRa3koHZazesUNMiQviDpxA","/ip4/172.18.89.3/tcp/53735/ws/p2p/16Uiu2HAkupbkd82QosVzfeYQTuNy7CRa3koHZazesUNMiQviDpxA"],"enrUri":"enr:-L24QDxvgBHS9oscbaBE7RhauOHsGtvyO0r8q_ewiMKp920ZT-60Ly5t9HrweX8PeM1Hefs3FT9U8qbKIU0z3Mz5fugCgmlkgnY0gmlwhKwSWQOKbXVsdGlhZGRyc5YACASsElkDBtHmAAoErBJZAwbR590DgnJzhQADAQAAiXNlY3AyNTZrMaECBd-UPH9BwCnPoL85B87lNFMoi0HYT6_2DHvMniPboeuDdGNwgtHmg3VkcILR6IV3YWt1MgE"}'
2026-03-25 04:38:03.937 INFO [src.node.waku_node] REST service is ready !!
2026-03-25 04:38:03.937 INFO [tests.e2e.test_network_conditions] Subscribing all nodes to relay topic
2026-03-25 04:38:03.937 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:24687/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-25 04:38:03.943 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-25 04:38:03.943 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:27795/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-25 04:38:03.950 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-25 04:38:03.951 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:9104/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-25 04:38:03.962 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-25 04:38:03.962 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16599/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-25 04:38:03.971 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-25 04:38:03.973 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19484/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-25 04:38:03.985 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-25 04:38:03.985 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:53302/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-25 04:38:03.993 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-25 04:38:03.997 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:53733/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-25 04:38:04.003 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-25 04:38:04.003 INFO [tests.e2e.test_network_conditions] Waiting for autoconnection
2026-03-25 04:38:04.003 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24687/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:04.007 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.224.238/tcp/49794/p2p/16Uiu2HAmRDGAeE14SYfhGiTJW91tcrNg41wRcwR3sCEhB8rvAX7E","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-11-g6a20ee","origin":"Discv5"}]'
2026-03-25 04:38:04.007 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27795/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:04.010 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.44.98/tcp/24688/p2p/16Uiu2HAmLYWdkLehMMU7pKPPA1bz2SBautR2rVagAzj45oALk82m","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-11-g6a20ee","origin":"Discv5"},{"multiaddr":"/ip4/172.18.191.6/tcp/47450/p2p/16Uiu2HAkxzqMvYZD9sDn9UkUX8CGkChU8n3paYwT4TGL8SZ5KXqv","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-11-g6a20ee","origin":"UnknownOrigin"}]'
2026-03-25 04:38:04.011 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9104/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:04.013 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.177.170/tcp/48004/p2p/16Uiu2HAmDDnRdyBGv7nxYuDzuU44deWLPKYZfbv9DscS6c9U17Rx","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-11-g6a20ee","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.224.238/tcp/27796/p2p/16Uiu2HAmRDGAeE14SYfhGiTJW91tcrNg41wRcwR3sCEhB8rvAX7E","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-11-g6a20ee","origin":"Discv5"}]'
2026-03-25 04:38:04.013 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16599/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:04.016 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.203.235/tcp/45030/p2p/16Uiu2HAmAcgqY5a8PvqGgSkyg74hc86xcZkRMoEWcdjJYte1yd8w","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-11-g6a20ee","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.191.6/tcp/9105/p2p/16Uiu2HAkxzqMvYZD9sDn9UkUX8CGkChU8n3paYwT4TGL8SZ5KXqv","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-11-g6a20ee","origin":"Discv5"}]'
2026-03-25 04:38:04.016 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19484/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:04.018 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.9.54/tcp/37838/p2p/16Uiu2HAm5DoJUpeeVW8TEH9Wc7Myv865pzk25feVr4hAQG2f2ZyW","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-11-g6a20ee","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.177.170/tcp/16600/p2p/16Uiu2HAmDDnRdyBGv7nxYuDzuU44deWLPKYZfbv9DscS6c9U17Rx","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-11-g6a20ee","origin":"Discv5"}]'
2026-03-25 04:38:04.019 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53302/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:04.021 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.203.235/tcp/19485/p2p/16Uiu2HAmAcgqY5a8PvqGgSkyg74hc86xcZkRMoEWcdjJYte1yd8w","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-11-g6a20ee","origin":"Discv5"},{"multiaddr":"/ip4/172.18.89.3/tcp/57580/p2p/16Uiu2HAkupbkd82QosVzfeYQTuNy7CRa3koHZazesUNMiQviDpxA","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-11-g6a20ee","origin":"UnknownOrigin"}]'
2026-03-25 04:38:04.021 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53733/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:38:04.024 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.9.54/tcp/53303/p2p/16Uiu2HAm5DoJUpeeVW8TEH9Wc7Myv865pzk25feVr4hAQG2f2ZyW","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-11-g6a20ee","origin":"Discv5"}]'
2026-03-25 04:38:04.024 DEBUG [src.libs.common] Sleeping for 60 seconds
2026-03-25 04:39:04.025 INFO [tests.e2e.test_network_conditions] Applying 3s latency to node3
2026-03-25 04:39:04.027 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '5465', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
2026-03-25 04:39:04.084 INFO [src.steps.network_conditions] TC exec: ['sudo', '-n', 'nsenter', '-t', '5465', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'delay', '3000ms']
2026-03-25 04:39:04.098 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9104/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-25 04:39:10.102 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[]'
2026-03-25 04:39:10.102 INFO [tests.e2e.test_network_conditions] Observed GET latency on node3: 6.00s
2026-03-25 04:39:10.104 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-25 04:39:10.105 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-25 04:39:10.105 DEBUG [src.node.waku_node] Stopping container with id b4d53847e13f
2026-03-25 04:39:10.713 DEBUG [src.node.waku_node] Container stopped.
2026-03-25 04:39:10.722 DEBUG [src.node.waku_node] Stopping container with id c9abaa387fcc
2026-03-25 04:39:11.343 DEBUG [src.node.waku_node] Container stopped.
2026-03-25 04:39:11.344 DEBUG [src.node.waku_node] Stopping container with id 7316a32f416d
2026-03-25 04:39:11.867 ERROR [src.node.docker_mananger] Max retries reached for container b4d53847e13f. Exiting log stream.
2026-03-25 04:39:11.949 DEBUG [src.node.waku_node] Container stopped.
2026-03-25 04:39:11.949 DEBUG [src.node.waku_node] Stopping container with id 5c691cd12316
2026-03-25 04:39:12.488 ERROR [src.node.docker_mananger] Max retries reached for container c9abaa387fcc. Exiting log stream.
2026-03-25 04:39:12.539 DEBUG [src.node.waku_node] Container stopped.
2026-03-25 04:39:12.541 DEBUG [src.node.waku_node] Stopping container with id 13c177992cda
2026-03-25 04:39:12.776 ERROR [src.node.docker_mananger] Max retries reached for container 7316a32f416d. Exiting log stream.
2026-03-25 04:39:13.114 DEBUG [src.node.waku_node] Container stopped.
2026-03-25 04:39:13.116 DEBUG [src.node.waku_node] Stopping container with id e6fb9a96abfc
2026-03-25 04:39:13.680 ERROR [src.node.docker_mananger] Max retries reached for container 5c691cd12316. Exiting log stream.
2026-03-25 04:39:13.682 DEBUG [src.node.waku_node] Container stopped.
2026-03-25 04:39:13.684 DEBUG [src.node.waku_node] Stopping container with id 60191545f466
2026-03-25 04:39:13.926 ERROR [src.node.docker_mananger] Max retries reached for container 13c177992cda. Exiting log stream.
2026-03-25 04:39:14.246 DEBUG [src.node.waku_node] Container stopped.
2026-03-25 04:39:14.247 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-25 04:39:14.423 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-25 04:39:14.589 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-25 04:39:14.670 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-25 04:39:14.778 ERROR [src.node.docker_mananger] Max retries reached for container e6fb9a96abfc. Exiting log stream.
2026-03-25 04:39:14.842 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-25 04:39:14.925 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-25 04:39:15.008 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-25 04:39:15.087 DEBUG [src.node.docker_mananger] No errors found in the waku logs.