2026-04-22 19:42:39 +00:00

192 lines
34 KiB
Plaintext

2026-04-22 19:39:43.311 DEBUG [tests.conftest] Running fixture setup: test_id
2026-04-22 19:39:43.312 DEBUG [tests.conftest] Running test: test_last_node_to_start_publishes with id: 2026-04-22_19-39-43__8dada304-40b8-433b-8106-dda25c26a292
2026-04-22 19:39:43.312 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-04-22 19:39:43.313 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-04-22 19:39:43.313 DEBUG [src.steps.relay] Running fixture setup: setup_main_relay_nodes
2026-04-22 19:39:43.321 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-22 19:39:43.322 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-04-22_19-39-43__8dada304-40b8-433b-8106-dda25c26a292__wakuorg_nwaku:latest.log
2026-04-22 19:39:43.322 DEBUG [src.node.waku_node] Starting Node...
2026-04-22 19:39:43.322 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-22 19:39:43.324 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-22 19:39:43.324 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.169.248
2026-04-22 19:39:43.325 DEBUG [src.node.docker_mananger] Generated ports ['13053', '13054', '13055', '13056', '13057']
2026-04-22 19:39:43.325 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-22 19:39:43.325 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-22 19:39:43.325 DEBUG [src.node.waku_node] Using volumes []
2026-04-22 19:39:43.326 DEBUG [src.node.docker_mananger] docker run -i -t -p 13053:13053 -p 13054:13054 -p 13055:13055 -p 13056:13056 -p 13057:13057 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=13055 --rest-port=13053 --tcp-port=13054 --discv5-udp-port=13056 --rest-address=0.0.0.0 --nat=extip:172.18.169.248 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=8f5e7ddaf8b67f18a6cf1da3dae7de2af2851bf98aea68d5e7cbf040e20ceafc --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=13057 --metrics-logging=true --relay=true
2026-04-22 19:39:43.515 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.169.248 waku f1248f37bb28698e21d50478d4802bbfa8fa8c96dd6eaa2e565f39747416cacd
2026-04-22 19:39:43.552 DEBUG [src.node.docker_mananger] Container started with ID f1248f37bb28. Setting up logs at ./log/docker/node1_2026-04-22_19-39-43__8dada304-40b8-433b-8106-dda25c26a292__wakuorg_nwaku:latest.log
2026-04-22 19:39:43.552 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 13053
2026-04-22 19:39:43.552 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-22 19:39:43.654 ERROR [src.node.docker_mananger] Max retries reached for container 523ceb1f2561. Exiting log stream.
2026-04-22 19:39:44.142 ERROR [src.node.docker_mananger] Max retries reached for container 4da3f33d0025. Exiting log stream.
2026-04-22 19:39:44.553 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13053/health" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:44.556 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-22 19:39:44.556 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-22 19:39:44.557 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13053/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:44.559 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.169.248/tcp/13054/p2p/16Uiu2HAmLe2Gg7X8K5KVbV5beLBQD77KiDekTB41PFDQhTLnciLd","/ip4/172.18.169.248/tcp/13055/ws/p2p/16Uiu2HAmLe2Gg7X8K5KVbV5beLBQD77KiDekTB41PFDQhTLnciLd"],"enrUri":"enr:-L24QJpuouC9pVayD3sexwrs47N-GcqQAiFNgaobk_agDC8YMYR9GM-Ml2Fq1GOsuxE1Iol6L6vd4desKRxmXnzqZ6kCgmlkgnY0gmlwhKwSqfiKbXVsdGlhZGRyc5YACASsEqn4BjL-AAoErBKp-AYy_90DgnJzhQDGAQAAiXNlY3AyNTZrMaEDdp4WBxfX0d1GcM3xk1XMbqvrIIDJ_d_bC7YahjhINt6DdGNwgjL-g3VkcIIzAIV3YWt1MgE"}'
2026-04-22 19:39:44.560 INFO [src.node.waku_node] REST service is ready !!
2026-04-22 19:39:44.567 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-22 19:39:44.567 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-04-22_19-39-43__8dada304-40b8-433b-8106-dda25c26a292__wakuorg_nwaku:latest.log
2026-04-22 19:39:44.567 DEBUG [src.node.waku_node] Starting Node...
2026-04-22 19:39:44.567 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-22 19:39:44.569 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-22 19:39:44.569 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.116.104
2026-04-22 19:39:44.569 DEBUG [src.node.docker_mananger] Generated ports ['15094', '15095', '15096', '15097', '15098']
2026-04-22 19:39:44.569 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-22 19:39:44.569 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-22 19:39:44.569 DEBUG [src.node.waku_node] Using volumes []
2026-04-22 19:39:44.570 DEBUG [src.node.docker_mananger] docker run -i -t -p 15094:15094 -p 15095:15095 -p 15096:15096 -p 15097:15097 -p 15098:15098 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=15096 --rest-port=15094 --tcp-port=15095 --discv5-udp-port=15097 --rest-address=0.0.0.0 --nat=extip:172.18.116.104 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=3f2f41ba157c07b4aaed1daf22d03efa239cc5f3a58c89ffc11a75b7aa5a2da3 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=15098 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QJpuouC9pVayD3sexwrs47N-GcqQAiFNgaobk_agDC8YMYR9GM-Ml2Fq1GOsuxE1Iol6L6vd4desKRxmXnzqZ6kCgmlkgnY0gmlwhKwSqfiKbXVsdGlhZGRyc5YACASsEqn4BjL-AAoErBKp-AYy_90DgnJzhQDGAQAAiXNlY3AyNTZrMaEDdp4WBxfX0d1GcM3xk1XMbqvrIIDJ_d_bC7YahjhINt6DdGNwgjL-g3VkcIIzAIV3YWt1MgE
2026-04-22 19:39:44.765 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.116.104 waku e5e3ac1c4cb715b7b23e64db3d655220e9b9fca0c70c03b263ddfeca2041b808
2026-04-22 19:39:44.801 DEBUG [src.node.docker_mananger] Container started with ID e5e3ac1c4cb7. Setting up logs at ./log/docker/node2_2026-04-22_19-39-43__8dada304-40b8-433b-8106-dda25c26a292__wakuorg_nwaku:latest.log
2026-04-22 19:39:44.802 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 15094
2026-04-22 19:39:44.802 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-22 19:39:45.803 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15094/health" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:45.810 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-22 19:39:45.810 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-22 19:39:45.811 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15094/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:45.814 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.116.104/tcp/15095/p2p/16Uiu2HAmPAqE4aLHzYQNiRQE3SYviGevxb8hToEDKQuTvoSsXCEc","/ip4/172.18.116.104/tcp/15096/ws/p2p/16Uiu2HAmPAqE4aLHzYQNiRQE3SYviGevxb8hToEDKQuTvoSsXCEc"],"enrUri":"enr:-L24QOMqleQ2NAB3KC_eJxG0iyOuOcbWODtk2KpK6jDxJwUAKWGrLCptUXT71Z5G7qHNg4A2YXC9QC5wQ-F4ET0NpTECgmlkgnY0gmlwhKwSdGiKbXVsdGlhZGRyc5YACASsEnRoBjr3AAoErBJ0aAY6-N0DgnJzhQDGAQAAiXNlY3AyNTZrMaEDnDn57ydJ2666WHDiV8YJoWJTWiA7L9Dr9YKA2u1sKBGDdGNwgjr3g3VkcII6-YV3YWt1MgE"}'
2026-04-22 19:39:45.816 INFO [src.node.waku_node] REST service is ready !!
2026-04-22 19:39:45.816 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15094/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.169.248/tcp/13054/p2p/16Uiu2HAmLe2Gg7X8K5KVbV5beLBQD77KiDekTB41PFDQhTLnciLd"]'
2026-04-22 19:39:45.822 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-22 19:39:45.822 DEBUG [src.steps.relay] Running fixture setup: setup_optional_relay_nodes
2026-04-22 19:39:45.829 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-22 19:39:45.829 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-04-22_19-39-43__8dada304-40b8-433b-8106-dda25c26a292__wakuorg_nwaku:latest.log
2026-04-22 19:39:45.829 DEBUG [src.node.waku_node] Starting Node...
2026-04-22 19:39:45.829 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-22 19:39:45.831 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-22 19:39:45.831 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.47.94
2026-04-22 19:39:45.831 DEBUG [src.node.docker_mananger] Generated ports ['15058', '15059', '15060', '15061', '15062']
2026-04-22 19:39:45.831 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-22 19:39:45.831 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-22 19:39:45.831 DEBUG [src.node.waku_node] Using volumes []
2026-04-22 19:39:45.832 DEBUG [src.node.docker_mananger] docker run -i -t -p 15058:15058 -p 15059:15059 -p 15060:15060 -p 15061:15061 -p 15062:15062 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=15060 --rest-port=15058 --tcp-port=15059 --discv5-udp-port=15061 --rest-address=0.0.0.0 --nat=extip:172.18.47.94 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=3580af2be2276dd791ca3ff3f02aeb059de95b34d0c57a044be10fa8d43d91c8 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=15062 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QJpuouC9pVayD3sexwrs47N-GcqQAiFNgaobk_agDC8YMYR9GM-Ml2Fq1GOsuxE1Iol6L6vd4desKRxmXnzqZ6kCgmlkgnY0gmlwhKwSqfiKbXVsdGlhZGRyc5YACASsEqn4BjL-AAoErBKp-AYy_90DgnJzhQDGAQAAiXNlY3AyNTZrMaEDdp4WBxfX0d1GcM3xk1XMbqvrIIDJ_d_bC7YahjhINt6DdGNwgjL-g3VkcIIzAIV3YWt1MgE
2026-04-22 19:39:46.013 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.47.94 waku 05e9e1b2b66848629ee45d57b2be312f84987ecab879fd10a42936d93c691081
2026-04-22 19:39:46.050 DEBUG [src.node.docker_mananger] Container started with ID 05e9e1b2b668. Setting up logs at ./log/docker/node3_2026-04-22_19-39-43__8dada304-40b8-433b-8106-dda25c26a292__wakuorg_nwaku:latest.log
2026-04-22 19:39:46.051 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 15058
2026-04-22 19:39:46.051 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-22 19:39:47.054 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15058/health" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:47.059 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-22 19:39:47.059 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-22 19:39:47.060 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15058/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:47.065 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.47.94/tcp/15059/p2p/16Uiu2HAmNPq5NZHg2EbNedPJN9EEabjHXxfbnhgm4xHWmFScB64v","/ip4/172.18.47.94/tcp/15060/ws/p2p/16Uiu2HAmNPq5NZHg2EbNedPJN9EEabjHXxfbnhgm4xHWmFScB64v"],"enrUri":"enr:-L24QNjXxOvyO6oq6Pi2ppnGyucCzkrKY1q1-OS1Ffk4tKGcA1Xigk-7kRTmtPDfkx7WS9JiXgIEwb8OXySmAflbbRoCgmlkgnY0gmlwhKwSL16KbXVsdGlhZGRyc5YACASsEi9eBjrTAAoErBIvXgY61N0DgnJzhQDGAQAAiXNlY3AyNTZrMaEDkLKrpQuO9Jh4epGuiTcbOBh44MKmMdHNaEAVbGiVhHeDdGNwgjrTg3VkcII61YV3YWt1MgE"}'
2026-04-22 19:39:47.066 INFO [src.node.waku_node] REST service is ready !!
2026-04-22 19:39:47.066 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15058/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.169.248/tcp/13054/p2p/16Uiu2HAmLe2Gg7X8K5KVbV5beLBQD77KiDekTB41PFDQhTLnciLd"]'
2026-04-22 19:39:47.069 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-22 19:39:47.076 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-22 19:39:47.076 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-04-22_19-39-43__8dada304-40b8-433b-8106-dda25c26a292__wakuorg_nwaku:latest.log
2026-04-22 19:39:47.076 DEBUG [src.node.waku_node] Starting Node...
2026-04-22 19:39:47.077 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-22 19:39:47.078 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-22 19:39:47.078 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.25.158
2026-04-22 19:39:47.078 DEBUG [src.node.docker_mananger] Generated ports ['28094', '28095', '28096', '28097', '28098']
2026-04-22 19:39:47.079 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-22 19:39:47.079 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-22 19:39:47.079 DEBUG [src.node.waku_node] Using volumes []
2026-04-22 19:39:47.079 DEBUG [src.node.docker_mananger] docker run -i -t -p 28094:28094 -p 28095:28095 -p 28096:28096 -p 28097:28097 -p 28098:28098 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=28096 --rest-port=28094 --tcp-port=28095 --discv5-udp-port=28097 --rest-address=0.0.0.0 --nat=extip:172.18.25.158 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=e5ef2f29ad907e3603dee673c51edf5cf105acaf26ef7b5d19c4dac717daceeb --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=28098 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QJpuouC9pVayD3sexwrs47N-GcqQAiFNgaobk_agDC8YMYR9GM-Ml2Fq1GOsuxE1Iol6L6vd4desKRxmXnzqZ6kCgmlkgnY0gmlwhKwSqfiKbXVsdGlhZGRyc5YACASsEqn4BjL-AAoErBKp-AYy_90DgnJzhQDGAQAAiXNlY3AyNTZrMaEDdp4WBxfX0d1GcM3xk1XMbqvrIIDJ_d_bC7YahjhINt6DdGNwgjL-g3VkcIIzAIV3YWt1MgE
2026-04-22 19:39:47.269 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.25.158 waku 0af9656c201e1e8fc5319617227826982510072516dd334c714034da355b9066
2026-04-22 19:39:47.308 DEBUG [src.node.docker_mananger] Container started with ID 0af9656c201e. Setting up logs at ./log/docker/node4_2026-04-22_19-39-43__8dada304-40b8-433b-8106-dda25c26a292__wakuorg_nwaku:latest.log
2026-04-22 19:39:47.308 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 28094
2026-04-22 19:39:47.308 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-22 19:39:48.309 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:28094/health" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:48.315 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-22 19:39:48.315 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-22 19:39:48.316 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:28094/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:48.322 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.25.158/tcp/28095/p2p/16Uiu2HAmG7QnJTDqg2ueTWH5TyfSfjnDoyKSYHGo9tPLXnQjLjC9","/ip4/172.18.25.158/tcp/28096/ws/p2p/16Uiu2HAmG7QnJTDqg2ueTWH5TyfSfjnDoyKSYHGo9tPLXnQjLjC9"],"enrUri":"enr:-L24QDQBXb5rKrBE11IL15dhM5QbAwUpVNZUcpExhumLe2H9VR8ew2irpob8zEOIcoSTis7p-VOd2GkVuqoR1ZQMigsCgmlkgnY0gmlwhKwSGZ6KbXVsdGlhZGRyc5YACASsEhmeBm2_AAoErBIZngZtwN0DgnJzhQDGAQAAiXNlY3AyNTZrMaEDM1fJS0FfxRSZ24559j8EYm8YMSPMjQBeksZF4taQwKaDdGNwgm2_g3VkcIJtwYV3YWt1MgE"}'
2026-04-22 19:39:48.323 INFO [src.node.waku_node] REST service is ready !!
2026-04-22 19:39:48.323 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:28094/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.169.248/tcp/13054/p2p/16Uiu2HAmLe2Gg7X8K5KVbV5beLBQD77KiDekTB41PFDQhTLnciLd"]'
2026-04-22 19:39:48.328 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-22 19:39:48.340 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-22 19:39:48.340 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-04-22_19-39-43__8dada304-40b8-433b-8106-dda25c26a292__wakuorg_nwaku:latest.log
2026-04-22 19:39:48.340 DEBUG [src.node.waku_node] Starting Node...
2026-04-22 19:39:48.340 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-22 19:39:48.342 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-22 19:39:48.342 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.217.46
2026-04-22 19:39:48.342 DEBUG [src.node.docker_mananger] Generated ports ['58806', '58807', '58808', '58809', '58810']
2026-04-22 19:39:48.343 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-22 19:39:48.343 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-22 19:39:48.343 DEBUG [src.node.waku_node] Using volumes []
2026-04-22 19:39:48.343 DEBUG [src.node.docker_mananger] docker run -i -t -p 58806:58806 -p 58807:58807 -p 58808:58808 -p 58809:58809 -p 58810:58810 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=58808 --rest-port=58806 --tcp-port=58807 --discv5-udp-port=58809 --rest-address=0.0.0.0 --nat=extip:172.18.217.46 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=ffbd0dcd2ade75beeb3bfac7440f6a8ceb6cf18cb314b4ef9d0a83d0798db0cd --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=58810 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QJpuouC9pVayD3sexwrs47N-GcqQAiFNgaobk_agDC8YMYR9GM-Ml2Fq1GOsuxE1Iol6L6vd4desKRxmXnzqZ6kCgmlkgnY0gmlwhKwSqfiKbXVsdGlhZGRyc5YACASsEqn4BjL-AAoErBKp-AYy_90DgnJzhQDGAQAAiXNlY3AyNTZrMaEDdp4WBxfX0d1GcM3xk1XMbqvrIIDJ_d_bC7YahjhINt6DdGNwgjL-g3VkcIIzAIV3YWt1MgE
2026-04-22 19:39:48.543 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.217.46 waku 38c63dadbab580ec7295e2090f6a915e372d383819f612880fd60bb4f22c295e
2026-04-22 19:39:48.579 DEBUG [src.node.docker_mananger] Container started with ID 38c63dadbab5. Setting up logs at ./log/docker/node5_2026-04-22_19-39-43__8dada304-40b8-433b-8106-dda25c26a292__wakuorg_nwaku:latest.log
2026-04-22 19:39:48.580 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 58806
2026-04-22 19:39:48.580 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-22 19:39:49.580 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58806/health" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:49.588 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-22 19:39:49.588 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-22 19:39:49.589 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58806/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:49.593 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.217.46/tcp/58807/p2p/16Uiu2HAmML3VKU9UMPAAMYMNfmACRvD4uqsqnPcjEcoKvG7Eo5UZ","/ip4/172.18.217.46/tcp/58808/ws/p2p/16Uiu2HAmML3VKU9UMPAAMYMNfmACRvD4uqsqnPcjEcoKvG7Eo5UZ"],"enrUri":"enr:-L24QNrTUYr17jyO7u8fqO3vjGVIjviaZE50uPCY5xe9I8hbSepI34AVCAaroNM8OIYC0uqVEK_DyYTOWt2rYjG339MCgmlkgnY0gmlwhKwS2S6KbXVsdGlhZGRyc5YACASsEtkuBuW3AAoErBLZLgbluN0DgnJzhQDGAQAAiXNlY3AyNTZrMaEDgN6yE0v6U48A0rRhLkwarjnpvrmWeNwQ4mmDqw219c6DdGNwguW3g3VkcILluYV3YWt1MgE"}'
2026-04-22 19:39:49.593 INFO [src.node.waku_node] REST service is ready !!
2026-04-22 19:39:49.594 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58806/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.169.248/tcp/13054/p2p/16Uiu2HAmLe2Gg7X8K5KVbV5beLBQD77KiDekTB41PFDQhTLnciLd"]'
2026-04-22 19:39:49.599 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-22 19:39:49.599 DEBUG [src.steps.relay] Running fixture setup: subscribe_main_relay_nodes
2026-04-22 19:39:49.600 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13053/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/1"]'
2026-04-22 19:39:49.612 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-22 19:39:49.612 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15094/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/1"]'
2026-04-22 19:39:49.620 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-22 19:39:49.620 DEBUG [src.steps.relay] Running fixture setup: subscribe_optional_relay_nodes
2026-04-22 19:39:49.622 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15058/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/1"]'
2026-04-22 19:39:49.629 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-22 19:39:49.630 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:28094/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/1"]'
2026-04-22 19:39:49.635 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-22 19:39:49.635 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58806/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/1"]'
2026-04-22 19:39:49.642 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-22 19:39:49.645 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13053/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-04-22 19:39:49.656 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-22 19:39:49.656 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-04-22 19:39:49.756 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-04-22 19:39:49.757 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13053/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:49.759 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776886789645049088,"ephemeral":false,"proof":""}]'
2026-04-22 19:39:49.761 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-04-22 19:39:49.761 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15094/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:49.764 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776886789645049088,"ephemeral":false,"proof":""}]'
2026-04-22 19:39:49.765 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-04-22 19:39:49.765 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15058/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:49.768 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776886789645049088,"ephemeral":false,"proof":""}]'
2026-04-22 19:39:49.770 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-04-22 19:39:49.770 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:28094/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:49.772 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776886789645049088,"ephemeral":false,"proof":""}]'
2026-04-22 19:39:49.773 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-04-22 19:39:49.774 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58806/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:49.776 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776886789645049088,"ephemeral":false,"proof":""}]'
2026-04-22 19:39:49.778 INFO [src.steps.relay] WARM UP successful!!
2026-04-22 19:39:49.781 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58806/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-04-22 19:39:49.785 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-22 19:39:49.786 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-04-22 19:39:49.886 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-04-22 19:39:49.886 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13053/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:49.889 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776886789780966475,"ephemeral":false,"proof":""}]'
2026-04-22 19:39:49.891 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-04-22 19:39:49.891 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15094/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:49.893 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776886789780966475,"ephemeral":false,"proof":""}]'
2026-04-22 19:39:49.895 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-04-22 19:39:49.895 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15058/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:49.897 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776886789780966475,"ephemeral":false,"proof":""}]'
2026-04-22 19:39:49.898 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-04-22 19:39:49.898 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:28094/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:49.901 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776886789780966475,"ephemeral":false,"proof":""}]'
2026-04-22 19:39:49.902 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-04-22 19:39:49.902 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58806/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-22 19:39:49.904 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776886789780966475,"ephemeral":false,"proof":""}]'
2026-04-22 19:39:49.908 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-04-22 19:39:49.909 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-04-22 19:39:49.909 DEBUG [src.node.waku_node] Stopping container with id f1248f37bb28
2026-04-22 19:39:50.386 DEBUG [src.node.waku_node] Container stopped.
2026-04-22 19:39:50.387 DEBUG [src.node.waku_node] Stopping container with id e5e3ac1c4cb7
2026-04-22 19:39:50.883 DEBUG [src.node.waku_node] Container stopped.
2026-04-22 19:39:50.884 DEBUG [src.node.waku_node] Stopping container with id 05e9e1b2b668
2026-04-22 19:39:51.260 ERROR [src.node.docker_mananger] Max retries reached for container f1248f37bb28. Exiting log stream.
2026-04-22 19:39:51.337 DEBUG [src.node.waku_node] Container stopped.
2026-04-22 19:39:51.339 DEBUG [src.node.waku_node] Stopping container with id 0af9656c201e
2026-04-22 19:39:51.698 ERROR [src.node.docker_mananger] Max retries reached for container e5e3ac1c4cb7. Exiting log stream.
2026-04-22 19:39:51.784 DEBUG [src.node.waku_node] Container stopped.
2026-04-22 19:39:51.784 DEBUG [src.node.waku_node] Stopping container with id 38c63dadbab5
2026-04-22 19:39:52.150 ERROR [src.node.docker_mananger] Max retries reached for container 05e9e1b2b668. Exiting log stream.
2026-04-22 19:39:52.249 DEBUG [src.node.waku_node] Container stopped.
2026-04-22 19:39:52.251 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-04-22 19:39:52.290 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-22 19:39:52.297 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-22 19:39:52.304 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-22 19:39:52.311 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-22 19:39:52.318 DEBUG [src.node.docker_mananger] No errors found in the waku logs.