180 lines
32 KiB
Plaintext

2026-01-31 04:38:41.554 DEBUG [tests.conftest] Running fixture setup: test_id
2026-01-31 04:38:41.554 DEBUG [tests.conftest] Running test: test_static_shard_relay with id: 2026-01-31_04-38-41__91bd2cc7-f378-438d-a543-5c78df3991c8
2026-01-31 04:38:41.554 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-01-31 04:38:41.555 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-01-31 04:38:41.555 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-01-31 04:38:41.562 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-31 04:38:41.562 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-01-31_04-38-41__91bd2cc7-f378-438d-a543-5c78df3991c8__wakuorg_nwaku:latest.log
2026-01-31 04:38:41.562 DEBUG [src.node.waku_node] Starting Node...
2026-01-31 04:38:41.562 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-31 04:38:41.564 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-31 04:38:41.564 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.145.228
2026-01-31 04:38:41.564 DEBUG [src.node.docker_mananger] Generated ports ['58250', '58251', '58252', '58253', '58254']
2026-01-31 04:38:41.564 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-01-31 04:38:41.564 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-01-31 04:38:41.565 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-31 04:38:41.565 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-31 04:38:41.565 DEBUG [src.node.waku_node] Using volumes []
2026-01-31 04:38:41.565 DEBUG [src.node.docker_mananger] docker run -i -t -p 58250:58250 -p 58251:58251 -p 58252:58252 -p 58253:58253 -p 58254:58254 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=58252 --rest-port=58250 --tcp-port=58251 --discv5-udp-port=58253 --rest-address=0.0.0.0 --nat=extip:172.18.145.228 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=a0be7fac92a0bfdbcba94c566b3f4b7645a0233798ee1fc5a3bdb9cf7f8adc3e --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=58254 --metrics-logging=true --relay=true --filter=true
2026-01-31 04:38:41.747 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.145.228 waku 3b1b00fc996c7f40580d3378f1d30c85ef58f953cb3b97ae305bc87f33fff352
2026-01-31 04:38:41.779 DEBUG [src.node.docker_mananger] Container started with ID 3b1b00fc996c. Setting up logs at ./log/docker/node1_2026-01-31_04-38-41__91bd2cc7-f378-438d-a543-5c78df3991c8__wakuorg_nwaku:latest.log
2026-01-31 04:38:41.779 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 58250
2026-01-31 04:38:41.779 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-31 04:38:41.856 ERROR [src.node.docker_mananger] Max retries reached for container 914f1d4c5f16. Exiting log stream.
2026-01-31 04:38:42.361 ERROR [src.node.docker_mananger] Max retries reached for container d4cbec1a4376. Exiting log stream.
2026-01-31 04:38:42.781 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58250/health" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:42.784 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-31 04:38:42.784 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-31 04:38:42.784 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58250/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:42.786 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.145.228/tcp/58251/p2p/16Uiu2HAmGQFos2wGB2fmmATERgQRKhW9SWkcnx4SuLzaGKGtYBK6","/ip4/172.18.145.228/tcp/58252/ws/p2p/16Uiu2HAmGQFos2wGB2fmmATERgQRKhW9SWkcnx4SuLzaGKGtYBK6"],"enrUri":"enr:-L24QMrn3UxA7Z6xQCE3UherrnIC3rFilunYq1pyMCqAsLo8ehWz7XDiMMFsjYHTqFa16lcWiFIjpqBDyEMUJZN-320CgmlkgnY0gmlwhKwSkeSKbXVsdGlhZGRyc5YACASsEpHkBuOLAAoErBKR5AbjjN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDN6iDoM5l9xe8D8ismmI9wDH6Km5LE11DMTSt2hZeBemDdGNwguOLg3VkcILjjYV3YWt1MgU"}'
2026-01-31 04:38:42.786 INFO [src.node.waku_node] REST service is ready !!
2026-01-31 04:38:42.793 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-31 04:38:42.793 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-01-31_04-38-41__91bd2cc7-f378-438d-a543-5c78df3991c8__wakuorg_nwaku:latest.log
2026-01-31 04:38:42.793 DEBUG [src.node.waku_node] Starting Node...
2026-01-31 04:38:42.793 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-31 04:38:42.794 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-31 04:38:42.794 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.28.47
2026-01-31 04:38:42.794 DEBUG [src.node.docker_mananger] Generated ports ['13962', '13963', '13964', '13965', '13966']
2026-01-31 04:38:42.795 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-01-31 04:38:42.795 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-01-31 04:38:42.795 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-31 04:38:42.795 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-31 04:38:42.795 DEBUG [src.node.waku_node] Using volumes []
2026-01-31 04:38:42.795 DEBUG [src.node.docker_mananger] docker run -i -t -p 13962:13962 -p 13963:13963 -p 13964:13964 -p 13965:13965 -p 13966:13966 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=13964 --rest-port=13962 --tcp-port=13963 --discv5-udp-port=13965 --rest-address=0.0.0.0 --nat=extip:172.18.28.47 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=5925ccb865fba76fc4952891ef7eb312ed09d695a58bbdad8ed3d14bd8b67cbd --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=13966 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QMrn3UxA7Z6xQCE3UherrnIC3rFilunYq1pyMCqAsLo8ehWz7XDiMMFsjYHTqFa16lcWiFIjpqBDyEMUJZN-320CgmlkgnY0gmlwhKwSkeSKbXVsdGlhZGRyc5YACASsEpHkBuOLAAoErBKR5AbjjN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDN6iDoM5l9xe8D8ismmI9wDH6Km5LE11DMTSt2hZeBemDdGNwguOLg3VkcILjjYV3YWt1MgU
2026-01-31 04:38:42.972 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.28.47 waku 5af4242e0e15281f22d7c33245ad5309ea34aed67b82fc299a99bc50a5d380ae
2026-01-31 04:38:43.005 DEBUG [src.node.docker_mananger] Container started with ID 5af4242e0e15. Setting up logs at ./log/docker/node2_2026-01-31_04-38-41__91bd2cc7-f378-438d-a543-5c78df3991c8__wakuorg_nwaku:latest.log
2026-01-31 04:38:43.005 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 13962
2026-01-31 04:38:43.006 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-31 04:38:44.006 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13962/health" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:44.019 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"}]}'
2026-01-31 04:38:44.022 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-31 04:38:44.023 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13962/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:44.027 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.28.47/tcp/13963/p2p/16Uiu2HAmSgWyJ6WaFFAJh1y5RwAPa8SmCozkDxCRxRzZygsqJSc9","/ip4/172.18.28.47/tcp/13964/ws/p2p/16Uiu2HAmSgWyJ6WaFFAJh1y5RwAPa8SmCozkDxCRxRzZygsqJSc9"],"enrUri":"enr:-L24QOBmYu-W5y6Hz_WGbhugwuSb4E8X41FDxmfOFIyQ3hO2Ls3ESpcTROVrX31RNQBHtWeif0ooEvYJRYB0pHbGQA0CgmlkgnY0gmlwhKwSHC-KbXVsdGlhZGRyc5YACASsEhwvBjaLAAoErBIcLwY2jN0DgnJzhQACAQAAiXNlY3AyNTZrMaED0GfSIMwUETy8epTSk_64P6hYmCaVshQSybPmHBvwWyKDdGNwgjaLg3VkcII2jYV3YWt1MgE"}'
2026-01-31 04:38:44.027 INFO [src.node.waku_node] REST service is ready !!
2026-01-31 04:38:44.027 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13962/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.145.228/tcp/58251/p2p/16Uiu2HAmGQFos2wGB2fmmATERgQRKhW9SWkcnx4SuLzaGKGtYBK6"]'
2026-01-31 04:38:44.030 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-31 04:38:44.037 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-31 04:38:44.037 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-01-31_04-38-41__91bd2cc7-f378-438d-a543-5c78df3991c8__wakuorg_nwaku:latest.log
2026-01-31 04:38:44.037 DEBUG [src.node.waku_node] Starting Node...
2026-01-31 04:38:44.037 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-31 04:38:44.038 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-31 04:38:44.039 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.193.87
2026-01-31 04:38:44.039 DEBUG [src.node.docker_mananger] Generated ports ['35738', '35739', '35740', '35741', '35742']
2026-01-31 04:38:44.039 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-01-31 04:38:44.039 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-01-31 04:38:44.039 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-31 04:38:44.039 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-31 04:38:44.039 DEBUG [src.node.waku_node] Using volumes []
2026-01-31 04:38:44.040 DEBUG [src.node.docker_mananger] docker run -i -t -p 35738:35738 -p 35739:35739 -p 35740:35740 -p 35741:35741 -p 35742:35742 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=35740 --rest-port=35738 --tcp-port=35739 --discv5-udp-port=35741 --rest-address=0.0.0.0 --nat=extip:172.18.193.87 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=fb0cec4e8b549a83aaa46c63bdba0112525e763dba48a6bcb74dffa3e5a68e4d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=35742 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QMrn3UxA7Z6xQCE3UherrnIC3rFilunYq1pyMCqAsLo8ehWz7XDiMMFsjYHTqFa16lcWiFIjpqBDyEMUJZN-320CgmlkgnY0gmlwhKwSkeSKbXVsdGlhZGRyc5YACASsEpHkBuOLAAoErBKR5AbjjN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDN6iDoM5l9xe8D8ismmI9wDH6Km5LE11DMTSt2hZeBemDdGNwguOLg3VkcILjjYV3YWt1MgU
2026-01-31 04:38:44.231 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.193.87 waku f9680de9ada5978cedf9cc172a364d8b9c5946d12b0a47898eca58c52bd53cd6
2026-01-31 04:38:44.265 DEBUG [src.node.docker_mananger] Container started with ID f9680de9ada5. Setting up logs at ./log/docker/node3_2026-01-31_04-38-41__91bd2cc7-f378-438d-a543-5c78df3991c8__wakuorg_nwaku:latest.log
2026-01-31 04:38:44.266 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 35738
2026-01-31 04:38:44.266 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-31 04:38:45.266 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35738/health" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:45.277 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"}]}'
2026-01-31 04:38:45.279 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-31 04:38:45.279 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35738/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:45.284 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.193.87/tcp/35739/p2p/16Uiu2HAm3R7ct4gU7oE3BdBywnvq2A8sYKrP6XT2NTgdZPvmdzo2","/ip4/172.18.193.87/tcp/35740/ws/p2p/16Uiu2HAm3R7ct4gU7oE3BdBywnvq2A8sYKrP6XT2NTgdZPvmdzo2"],"enrUri":"enr:-L24QAk3MaHLV10-5WMt4rxCubtDPPOlz4a5uvxhlGQOYI1NGwkzHW87XS7BERvn3DnpLPlGNVcb3zkM5evtE3sdeBoCgmlkgnY0gmlwhKwSwVeKbXVsdGlhZGRyc5YACASsEsFXBoubAAoErBLBVwaLnN0DgnJzhQACAQAAiXNlY3AyNTZrMaECdrjnvAKfB8zEalVH5VEVa4iSJ4uL1K4y8g4B3NTqEXGDdGNwgoubg3VkcIKLnYV3YWt1MgE"}'
2026-01-31 04:38:45.284 INFO [src.node.waku_node] REST service is ready !!
2026-01-31 04:38:45.285 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35738/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.145.228/tcp/58251/p2p/16Uiu2HAmGQFos2wGB2fmmATERgQRKhW9SWkcnx4SuLzaGKGtYBK6"]'
2026-01-31 04:38:45.287 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-31 04:38:45.294 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-31 04:38:45.294 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-01-31_04-38-41__91bd2cc7-f378-438d-a543-5c78df3991c8__wakuorg_nwaku:latest.log
2026-01-31 04:38:45.294 DEBUG [src.node.waku_node] Starting Node...
2026-01-31 04:38:45.294 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-31 04:38:45.296 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-31 04:38:45.296 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.155.25
2026-01-31 04:38:45.296 DEBUG [src.node.docker_mananger] Generated ports ['43552', '43553', '43554', '43555', '43556']
2026-01-31 04:38:45.296 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-01-31 04:38:45.296 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-01-31 04:38:45.296 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-31 04:38:45.296 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-31 04:38:45.297 DEBUG [src.node.waku_node] Using volumes []
2026-01-31 04:38:45.297 DEBUG [src.node.docker_mananger] docker run -i -t -p 43552:43552 -p 43553:43553 -p 43554:43554 -p 43555:43555 -p 43556:43556 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=43554 --rest-port=43552 --tcp-port=43553 --discv5-udp-port=43555 --rest-address=0.0.0.0 --nat=extip:172.18.155.25 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=fde30de93fd4fc6982abd762a2eadfe9e47e5fcdfdadc0b01ec95937bf24e3be --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=43556 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QMrn3UxA7Z6xQCE3UherrnIC3rFilunYq1pyMCqAsLo8ehWz7XDiMMFsjYHTqFa16lcWiFIjpqBDyEMUJZN-320CgmlkgnY0gmlwhKwSkeSKbXVsdGlhZGRyc5YACASsEpHkBuOLAAoErBKR5AbjjN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDN6iDoM5l9xe8D8ismmI9wDH6Km5LE11DMTSt2hZeBemDdGNwguOLg3VkcILjjYV3YWt1MgU
2026-01-31 04:38:45.477 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.155.25 waku f24ed79453bb29830bf236c92725ffb8f4339fc31eee72982f04012a9d50f37a
2026-01-31 04:38:45.512 DEBUG [src.node.docker_mananger] Container started with ID f24ed79453bb. Setting up logs at ./log/docker/node4_2026-01-31_04-38-41__91bd2cc7-f378-438d-a543-5c78df3991c8__wakuorg_nwaku:latest.log
2026-01-31 04:38:45.513 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 43552
2026-01-31 04:38:45.513 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-31 04:38:46.513 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43552/health" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:46.521 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"READY"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"}]}'
2026-01-31 04:38:46.521 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-31 04:38:46.522 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43552/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:46.525 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.155.25/tcp/43553/p2p/16Uiu2HAmQbY7JzsDJzvTwafcKCHREzSs53bfdD22EG8pYJPFTRTN","/ip4/172.18.155.25/tcp/43554/ws/p2p/16Uiu2HAmQbY7JzsDJzvTwafcKCHREzSs53bfdD22EG8pYJPFTRTN"],"enrUri":"enr:-L24QE9w4sK9Rsg2GPEeVzY2SunKrvEpyzHkV2h-odWu2ZVGYouj8FRTXxPLsGe81z505VpAF9WHkgH4NGZDsnlBom8CgmlkgnY0gmlwhKwSmxmKbXVsdGlhZGRyc5YACASsEpsZBqohAAoErBKbGQaqIt0DgnJzhQACAQAAiXNlY3AyNTZrMaEDsWnUcxTCBcepCvHGDCJHK-kNfbbjoU6sM62_NV3uJMmDdGNwgqohg3VkcIKqI4V3YWt1MgE"}'
2026-01-31 04:38:46.525 INFO [src.node.waku_node] REST service is ready !!
2026-01-31 04:38:46.525 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:43552/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.145.228/tcp/58251/p2p/16Uiu2HAmGQFos2wGB2fmmATERgQRKhW9SWkcnx4SuLzaGKGtYBK6"]'
2026-01-31 04:38:46.528 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-31 04:38:46.534 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-31 04:38:46.534 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-01-31_04-38-41__91bd2cc7-f378-438d-a543-5c78df3991c8__wakuorg_nwaku:latest.log
2026-01-31 04:38:46.534 DEBUG [src.node.waku_node] Starting Node...
2026-01-31 04:38:46.535 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-31 04:38:46.536 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-31 04:38:46.536 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.64.110
2026-01-31 04:38:46.536 DEBUG [src.node.docker_mananger] Generated ports ['48944', '48945', '48946', '48947', '48948']
2026-01-31 04:38:46.536 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-01-31 04:38:46.536 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-01-31 04:38:46.537 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-31 04:38:46.537 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-31 04:38:46.537 DEBUG [src.node.waku_node] Using volumes []
2026-01-31 04:38:46.537 DEBUG [src.node.docker_mananger] docker run -i -t -p 48944:48944 -p 48945:48945 -p 48946:48946 -p 48947:48947 -p 48948:48948 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=48946 --rest-port=48944 --tcp-port=48945 --discv5-udp-port=48947 --rest-address=0.0.0.0 --nat=extip:172.18.64.110 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=856a4cef9f6af4486a00d22cebb92bdada750b02a5dc974c34870d13f370ef0f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=48948 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QMrn3UxA7Z6xQCE3UherrnIC3rFilunYq1pyMCqAsLo8ehWz7XDiMMFsjYHTqFa16lcWiFIjpqBDyEMUJZN-320CgmlkgnY0gmlwhKwSkeSKbXVsdGlhZGRyc5YACASsEpHkBuOLAAoErBKR5AbjjN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDN6iDoM5l9xe8D8ismmI9wDH6Km5LE11DMTSt2hZeBemDdGNwguOLg3VkcILjjYV3YWt1MgU
2026-01-31 04:38:46.720 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.64.110 waku cb890bf7bad09371c2f46d86230c777a8b3da12c639019121f24483be989028a
2026-01-31 04:38:46.752 DEBUG [src.node.docker_mananger] Container started with ID cb890bf7bad0. Setting up logs at ./log/docker/node5_2026-01-31_04-38-41__91bd2cc7-f378-438d-a543-5c78df3991c8__wakuorg_nwaku:latest.log
2026-01-31 04:38:46.753 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 48944
2026-01-31 04:38:46.753 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-31 04:38:47.753 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:48944/health" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:47.763 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"}]}'
2026-01-31 04:38:47.764 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-31 04:38:47.764 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:48944/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:47.769 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.64.110/tcp/48945/p2p/16Uiu2HAm4DAVV33kNQebBS61TzQ4MQvL9nS2WJHcLy9YeQKqmsUT","/ip4/172.18.64.110/tcp/48946/ws/p2p/16Uiu2HAm4DAVV33kNQebBS61TzQ4MQvL9nS2WJHcLy9YeQKqmsUT"],"enrUri":"enr:-L24QGpDerPyS9-SfvQ08b-PO9YagbP9j_RS9nbdfuuvPcqSYteSpyXvuYJLoSMjEZEVjTL_kr_o1iKWh6Vpb93Wb7YCgmlkgnY0gmlwhKwSQG6KbXVsdGlhZGRyc5YACASsEkBuBr8xAAoErBJAbga_Mt0DgnJzhQACAQAAiXNlY3AyNTZrMaECgoTezl3H87w6ccof4orIzB5kVjUNvlNcGUlGjjLB-aCDdGNwgr8xg3VkcIK_M4V3YWt1MgE"}'
2026-01-31 04:38:47.769 INFO [src.node.waku_node] REST service is ready !!
2026-01-31 04:38:47.770 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48944/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.145.228/tcp/58251/p2p/16Uiu2HAmGQFos2wGB2fmmATERgQRKhW9SWkcnx4SuLzaGKGtYBK6"]'
2026-01-31 04:38:47.773 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-31 04:38:47.774 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58250/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-01-31 04:38:47.776 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-31 04:38:47.777 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13962/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-01-31 04:38:47.779 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-31 04:38:47.779 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35738/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-01-31 04:38:47.782 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-31 04:38:47.782 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:43552/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-01-31 04:38:47.784 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-31 04:38:47.785 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48944/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-01-31 04:38:47.787 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-31 04:38:47.787 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58250/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-31 04:38:47.795 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-31 04:38:47.795 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-01-31 04:38:47.896 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-01-31 04:38:47.896 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58250/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:47.898 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1769834327787740926,"ephemeral":false,"proof":""}]'
2026-01-31 04:38:47.900 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-01-31 04:38:47.900 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13962/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:47.902 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1769834327787740926,"ephemeral":false,"proof":""}]'
2026-01-31 04:38:47.903 DEBUG [src.steps.sharding] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-01-31 04:38:47.904 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35738/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:47.906 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1769834327787740926,"ephemeral":false,"proof":""}]'
2026-01-31 04:38:47.907 DEBUG [src.steps.sharding] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-01-31 04:38:47.907 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43552/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:47.909 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1769834327787740926,"ephemeral":false,"proof":""}]'
2026-01-31 04:38:47.910 DEBUG [src.steps.sharding] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-01-31 04:38:47.911 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:48944/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-01-31 04:38:47.913 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1769834327787740926,"ephemeral":false,"proof":""}]'
2026-01-31 04:38:47.915 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-01-31 04:38:47.916 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-01-31 04:38:47.917 DEBUG [src.node.waku_node] Stopping container with id 3b1b00fc996c
2026-01-31 04:38:48.508 DEBUG [src.node.waku_node] Container stopped.
2026-01-31 04:38:48.508 DEBUG [src.node.waku_node] Stopping container with id 5af4242e0e15
2026-01-31 04:38:49.071 DEBUG [src.node.waku_node] Container stopped.
2026-01-31 04:38:49.071 DEBUG [src.node.waku_node] Stopping container with id f9680de9ada5
2026-01-31 04:38:49.320 ERROR [src.node.docker_mananger] Max retries reached for container 3b1b00fc996c. Exiting log stream.
2026-01-31 04:38:49.636 DEBUG [src.node.waku_node] Container stopped.
2026-01-31 04:38:49.637 DEBUG [src.node.waku_node] Stopping container with id f24ed79453bb
2026-01-31 04:38:49.883 ERROR [src.node.docker_mananger] Max retries reached for container 5af4242e0e15. Exiting log stream.
2026-01-31 04:38:50.155 DEBUG [src.node.waku_node] Container stopped.
2026-01-31 04:38:50.157 DEBUG [src.node.waku_node] Stopping container with id cb890bf7bad0
2026-01-31 04:38:50.449 ERROR [src.node.docker_mananger] Max retries reached for container f9680de9ada5. Exiting log stream.
2026-01-31 04:38:50.656 DEBUG [src.node.waku_node] Container stopped.
2026-01-31 04:38:50.658 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-01-31 04:38:50.671 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-31 04:38:50.675 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-31 04:38:50.680 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-31 04:38:50.684 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-31 04:38:50.689 DEBUG [src.node.docker_mananger] No errors found in the waku logs.