270 lines
48 KiB
Plaintext

2026-03-18 04:49:41.487 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-18 04:49:41.487 DEBUG [tests.conftest] Running test: test_store_sync_long_chain with id: 2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2
2026-03-18 04:49:41.487 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-18 04:49:41.494 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:49:41.495 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:41.502 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:49:41.502 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:41.509 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:49:41.509 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:41.509 DEBUG [src.steps.store] Running fixture setup: store_setup
2026-03-18 04:49:41.510 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:49:41.510 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:49:41.512 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:49:41.512 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.2.100
2026-03-18 04:49:41.512 DEBUG [src.node.docker_mananger] Generated ports ['48861', '48862', '48863', '48864', '48865']
2026-03-18 04:49:41.512 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:49:41.512 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:49:41.513 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:49:41.513 DEBUG [src.node.docker_mananger] docker run -i -t -p 48861:48861 -p 48862:48862 -p 48863:48863 -p 48864:48864 -p 48865:48865 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=48863 --rest-port=48861 --tcp-port=48862 --discv5-udp-port=48864 --rest-address=0.0.0.0 --nat=extip:172.18.2.100 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=964a5edc2d6c7ed95dffbb450bae46baed1bf7dc74b750bdec00f635d9d33531 --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=48865 --metrics-logging=true --store-sync-relay-jitter=0 --relay=true --dns-discovery=false
2026-03-18 04:49:41.709 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.2.100 waku 63ab676e0ddcaae9de9e973af2a4d8b884f3336ca16e42bb50f6f970747acc27
2026-03-18 04:49:41.731 ERROR [src.node.docker_mananger] Max retries reached for container 10b947c65225. Exiting log stream.
2026-03-18 04:49:41.746 DEBUG [src.node.docker_mananger] Container started with ID 63ab676e0ddc. Setting up logs at ./log/docker/node1_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:41.747 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 48861
2026-03-18 04:49:41.747 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:49:42.280 ERROR [src.node.docker_mananger] Max retries reached for container d0234c79fe00. Exiting log stream.
2026-03-18 04:49:42.748 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:48861/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:42.751 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:49:42.751 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:49:42.751 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:48861/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:42.753 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.2.100/tcp/48862/p2p/16Uiu2HAmNNN8p1gEvrYqB2SAEA96LE4q4kt8aLyAwiJgUAi2RYmo","/ip4/172.18.2.100/tcp/48863/ws/p2p/16Uiu2HAmNNN8p1gEvrYqB2SAEA96LE4q4kt8aLyAwiJgUAi2RYmo"],"enrUri":"enr:-L24QNrxMBjNXNmpwbla949c-Hf32zwHdy_dIcHFOngnO8QALOhY-pH8w2hlXOe1UnQRHhIlHL-rF4F6OKCJWv_oMlMCgmlkgnY0gmlwhKwSAmSKbXVsdGlhZGRyc5YACASsEgJkBr7eAAoErBICZAa-390DgnJzhQADAQAAiXNlY3AyNTZrMaEDkFKgslfmsANNBekkJlNS8UQ-CHKcMz24BVuI2ny10bKDdGNwgr7eg3VkcIK-4IV3YWt1MhM"}'
2026-03-18 04:49:42.754 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:49:42.754 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48861/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-18 04:49:42.772 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:42.779 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:49:42.779 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:42.786 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:49:42.786 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:42.793 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:49:42.793 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node6_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:42.799 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:49:42.800 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node7_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:42.806 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:49:42.806 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node8_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:42.806 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:49:42.806 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:49:42.808 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:49:42.808 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.22.255
2026-03-18 04:49:42.808 DEBUG [src.node.docker_mananger] Generated ports ['27108', '27109', '27110', '27111', '27112']
2026-03-18 04:49:42.808 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:49:42.808 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:49:42.808 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:49:42.808 DEBUG [src.node.docker_mananger] docker run -i -t -p 27108:27108 -p 27109:27109 -p 27110:27110 -p 27111:27111 -p 27112:27112 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=27110 --rest-port=27108 --tcp-port=27109 --discv5-udp-port=27111 --rest-address=0.0.0.0 --nat=extip:172.18.22.255 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=1af31fb4cc7f6640edf135dbcccfb5e82a90462cebe85899b2fdddcf74bd1c3b --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=27112 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-18 04:49:43.017 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.22.255 waku 674ff9c1063a515bf18c943914ea04f0cc4f7924fdaaf776328ded527d4517dd
2026-03-18 04:49:43.058 DEBUG [src.node.docker_mananger] Container started with ID 674ff9c1063a. Setting up logs at ./log/docker/node2_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:43.058 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 27108
2026-03-18 04:49:43.059 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:49:44.060 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27108/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:44.062 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:49:44.063 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:49:44.063 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27108/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:44.065 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.22.255/tcp/27109/p2p/16Uiu2HAm7Zu16XT1Jj9Xb3d5UftKXZE4WRahDGjUPRPdQMNWzYn1","/ip4/172.18.22.255/tcp/27110/ws/p2p/16Uiu2HAm7Zu16XT1Jj9Xb3d5UftKXZE4WRahDGjUPRPdQMNWzYn1"],"enrUri":"enr:-L24QJE_QBWW4ETsgyRnj7sBE5iuLW0U1xDL2UvWdwamWxdia-gMwQatSZlOgp0TmdwYlI-JlJyp1wICMbhh_-yK02oCgmlkgnY0gmlwhKwSFv-KbXVsdGlhZGRyc5YACASsEhb_BmnlAAoErBIW_wZp5t0DgnJzhQADAQAAiXNlY3AyNTZrMaECtGedN41MhkyWDaL_GhIodw3e6DZdXoXKoBC16W1PQ-aDdGNwgmnlg3VkcIJp54V3YWt1MhI"}'
2026-03-18 04:49:44.066 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:49:44.066 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:27108/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.2.100/tcp/48862/p2p/16Uiu2HAmNNN8p1gEvrYqB2SAEA96LE4q4kt8aLyAwiJgUAi2RYmo"]'
2026-03-18 04:49:44.106 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:44.107 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:49:44.107 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:49:44.113 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:49:44.115 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.51.114
2026-03-18 04:49:44.116 DEBUG [src.node.docker_mananger] Generated ports ['20566', '20567', '20568', '20569', '20570']
2026-03-18 04:49:44.117 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:49:44.117 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:49:44.117 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:49:44.117 DEBUG [src.node.docker_mananger] docker run -i -t -p 20566:20566 -p 20567:20567 -p 20568:20568 -p 20569:20569 -p 20570:20570 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=20568 --rest-port=20566 --tcp-port=20567 --discv5-udp-port=20569 --rest-address=0.0.0.0 --nat=extip:172.18.51.114 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=59abcad6cbaf768afe4d9489383e3c8eb99eda9cb0ba8abe5a2a97d261aef6ce --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=20570 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-18 04:49:44.327 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.51.114 waku 38c94748a3c1bedd81b7508e420a17eacf68d753e268fc12eeb1a7e5dc9808de
2026-03-18 04:49:44.362 DEBUG [src.node.docker_mananger] Container started with ID 38c94748a3c1. Setting up logs at ./log/docker/node3_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:44.363 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 20566
2026-03-18 04:49:44.363 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:49:45.364 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:20566/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:45.366 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:49:45.367 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:49:45.367 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:20566/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:45.369 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.51.114/tcp/20567/p2p/16Uiu2HAmPBzAN476T3PiQ2k6vbunExhDdeB8xDWMSTivapAEjj2v","/ip4/172.18.51.114/tcp/20568/ws/p2p/16Uiu2HAmPBzAN476T3PiQ2k6vbunExhDdeB8xDWMSTivapAEjj2v"],"enrUri":"enr:-L24QLH9HgIlsVJvaZdAIB4pImkxg_8eMY5jC7Umgo1vw23LKZTaaqUBeydI8RC0j_U9zeYmMLZ6H1KsH0JkCHq1e6YCgmlkgnY0gmlwhKwSM3KKbXVsdGlhZGRyc5YACASsEjNyBlBXAAoErBIzcgZQWN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDnIWpTbu5XdPd_Eth_o_9eqzbgWd691GVkcUJvHLsLpeDdGNwglBXg3VkcIJQWYV3YWt1MhI"}'
2026-03-18 04:49:45.369 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:49:45.370 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:20566/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.22.255/tcp/27109/p2p/16Uiu2HAm7Zu16XT1Jj9Xb3d5UftKXZE4WRahDGjUPRPdQMNWzYn1"]'
2026-03-18 04:49:45.405 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:45.405 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:49:45.406 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:49:45.411 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:49:45.412 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.84.248
2026-03-18 04:49:45.412 DEBUG [src.node.docker_mananger] Generated ports ['11937', '11938', '11939', '11940', '11941']
2026-03-18 04:49:45.413 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:49:45.413 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:49:45.413 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:49:45.415 DEBUG [src.node.docker_mananger] docker run -i -t -p 11937:11937 -p 11938:11938 -p 11939:11939 -p 11940:11940 -p 11941:11941 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=11939 --rest-port=11937 --tcp-port=11938 --discv5-udp-port=11940 --rest-address=0.0.0.0 --nat=extip:172.18.84.248 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=dc59b48b1aedde6cbb02a0c31c02106ab71ac5239c0ded4ce6b2b11bccee788e --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=11941 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-18 04:49:45.614 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.84.248 waku c153586597e137134c8f75d2349cd35ec9ecb2130cd431f24fdf4161941b2eec
2026-03-18 04:49:45.648 DEBUG [src.node.docker_mananger] Container started with ID c153586597e1. Setting up logs at ./log/docker/node4_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:45.648 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 11937
2026-03-18 04:49:45.648 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:49:46.649 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:11937/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:46.652 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:49:46.652 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:49:46.652 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:11937/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:46.655 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.84.248/tcp/11938/p2p/16Uiu2HAm8QMiw12V4jq3cHtSahxicP7bm6PZVpcjTFmBfamEDQhn","/ip4/172.18.84.248/tcp/11939/ws/p2p/16Uiu2HAm8QMiw12V4jq3cHtSahxicP7bm6PZVpcjTFmBfamEDQhn"],"enrUri":"enr:-L24QKnlFHh68sF2JoqM-KBJQzq7fKyOvx_i30pGDBVGVSK2LBE29367JbCM_1xxRITWzLNzPSGP9EJWxCd92r1qjJ8CgmlkgnY0gmlwhKwSVPiKbXVsdGlhZGRyc5YACASsElT4Bi6iAAoErBJU-AYuo90DgnJzhQADAQAAiXNlY3AyNTZrMaECwNG085yz4kzrQ_f3uFDHYAD4ewZuL8MmO7u-a2P4uemDdGNwgi6ig3VkcIIupIV3YWt1MhI"}'
2026-03-18 04:49:46.655 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:49:46.655 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:11937/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.51.114/tcp/20567/p2p/16Uiu2HAmPBzAN476T3PiQ2k6vbunExhDdeB8xDWMSTivapAEjj2v"]'
2026-03-18 04:49:46.691 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:46.692 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:49:46.693 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:49:46.699 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:49:46.699 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.138.167
2026-03-18 04:49:46.700 DEBUG [src.node.docker_mananger] Generated ports ['58544', '58545', '58546', '58547', '58548']
2026-03-18 04:49:46.700 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:49:46.700 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:49:46.701 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:49:46.701 DEBUG [src.node.docker_mananger] docker run -i -t -p 58544:58544 -p 58545:58545 -p 58546:58546 -p 58547:58547 -p 58548:58548 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=58546 --rest-port=58544 --tcp-port=58545 --discv5-udp-port=58547 --rest-address=0.0.0.0 --nat=extip:172.18.138.167 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=af5d96265ed8a7485a8af0efdec5c4bb6da811bbbdd83fdcfc33dd6bcddbb0eb --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=58548 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-18 04:49:46.908 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.138.167 waku f222720161178b6ee6906792edd8fb3de9a7bd876050d25bb80868d3ce9dae7c
2026-03-18 04:49:46.946 DEBUG [src.node.docker_mananger] Container started with ID f22272016117. Setting up logs at ./log/docker/node5_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:46.946 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 58544
2026-03-18 04:49:46.947 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:49:47.948 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58544/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:47.951 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:49:47.951 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:49:47.951 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58544/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:47.954 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.138.167/tcp/58545/p2p/16Uiu2HAmK1CwLbZv6LSoPiH7LBvhEyKGL1coc1Eae6xZBxQa5E1S","/ip4/172.18.138.167/tcp/58546/ws/p2p/16Uiu2HAmK1CwLbZv6LSoPiH7LBvhEyKGL1coc1Eae6xZBxQa5E1S"],"enrUri":"enr:-L24QBywkuJweuZHmHQl1za2eXDM1AWktHtfSNB-CGiMFCr6X_OMvAocr-wfPZ3rUNlst8z9-gPFaXgbncDUvk5r-8ECgmlkgnY0gmlwhKwSiqeKbXVsdGlhZGRyc5YACASsEoqnBuSxAAoErBKKpwbkst0DgnJzhQADAQAAiXNlY3AyNTZrMaEDXlP4T1Ufu7UsS9KZ4k-oaz7OAtiz_OrcfqFEw_nIlr2DdGNwguSxg3VkcILks4V3YWt1MhI"}'
2026-03-18 04:49:47.954 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:49:47.954 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58544/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.84.248/tcp/11938/p2p/16Uiu2HAm8QMiw12V4jq3cHtSahxicP7bm6PZVpcjTFmBfamEDQhn"]'
2026-03-18 04:49:47.985 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:47.986 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:49:47.986 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:49:47.991 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:49:47.994 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.107.63
2026-03-18 04:49:47.995 DEBUG [src.node.docker_mananger] Generated ports ['16951', '16952', '16953', '16954', '16955']
2026-03-18 04:49:47.995 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:49:47.996 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:49:47.996 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:49:47.997 DEBUG [src.node.docker_mananger] docker run -i -t -p 16951:16951 -p 16952:16952 -p 16953:16953 -p 16954:16954 -p 16955:16955 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=16953 --rest-port=16951 --tcp-port=16952 --discv5-udp-port=16954 --rest-address=0.0.0.0 --nat=extip:172.18.107.63 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=e8fb7ddba724a3bd3c9cd5ddd5f14ca4d68e22b9ad5ccd01c893ed4fb9afb5d8 --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=16955 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-18 04:49:48.205 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.107.63 waku 1fe0f1cc4a41ce4dddc01e390b119ee17c205f537516b21a0c740d7521af0926
2026-03-18 04:49:48.241 DEBUG [src.node.docker_mananger] Container started with ID 1fe0f1cc4a41. Setting up logs at ./log/docker/node6_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:48.242 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 16951
2026-03-18 04:49:48.243 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:49:49.243 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16951/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:49.246 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:49:49.247 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:49:49.247 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16951/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:49.249 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.107.63/tcp/16952/p2p/16Uiu2HAmMmbSVEQZQTjFUwU9qAPpbNgN5faLkpevhZzhYayvbQ3P","/ip4/172.18.107.63/tcp/16953/ws/p2p/16Uiu2HAmMmbSVEQZQTjFUwU9qAPpbNgN5faLkpevhZzhYayvbQ3P"],"enrUri":"enr:-L24QHfEsXf246_A7vtYzoA4EuP2cXxSWFG8zN-Y-6EqMrrTM0OwnDarRTB9Hwz9_l_jINsfyLKf2r8tTnzgved9p0sCgmlkgnY0gmlwhKwSaz-KbXVsdGlhZGRyc5YACASsEms_BkI4AAoErBJrPwZCOd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDh2pX2GuxO254_sni8E5_iAXMgHkZgvYI-PltRsjxTSaDdGNwgkI4g3VkcIJCOoV3YWt1MhI"}'
2026-03-18 04:49:49.249 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:49:49.250 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16951/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.138.167/tcp/58545/p2p/16Uiu2HAmK1CwLbZv6LSoPiH7LBvhEyKGL1coc1Eae6xZBxQa5E1S"]'
2026-03-18 04:49:49.289 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:49.290 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:49:49.290 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:49:49.294 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:49:49.294 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.106.251
2026-03-18 04:49:49.294 DEBUG [src.node.docker_mananger] Generated ports ['40656', '40657', '40658', '40659', '40660']
2026-03-18 04:49:49.295 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:49:49.295 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:49:49.295 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:49:49.295 DEBUG [src.node.docker_mananger] docker run -i -t -p 40656:40656 -p 40657:40657 -p 40658:40658 -p 40659:40659 -p 40660:40660 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=40658 --rest-port=40656 --tcp-port=40657 --discv5-udp-port=40659 --rest-address=0.0.0.0 --nat=extip:172.18.106.251 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=5ebd4bb8cdcf455f932bfa2ec58fdd8bba7ce319bdbbb8db5abbfd2ae67b1d89 --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=40660 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-18 04:49:49.502 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.106.251 waku 9f40f809d825e52b0feaabff8c3fb855c7530764401eae0b00afabb02ccb4b2a
2026-03-18 04:49:49.538 DEBUG [src.node.docker_mananger] Container started with ID 9f40f809d825. Setting up logs at ./log/docker/node7_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:49.539 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 40656
2026-03-18 04:49:49.539 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:49:50.540 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:40656/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:50.543 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:49:50.543 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:49:50.543 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:40656/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:50.546 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.106.251/tcp/40657/p2p/16Uiu2HAm6MSW54pt6uBdcazZCvk7fdGuYZFrx1wPRkq1petVZ3Ha","/ip4/172.18.106.251/tcp/40658/ws/p2p/16Uiu2HAm6MSW54pt6uBdcazZCvk7fdGuYZFrx1wPRkq1petVZ3Ha"],"enrUri":"enr:-L24QPiIhsrrqxHRjvHlgxDxs5yd5FiLP2lK1cKo96skoMiRSPWKivMA7YU_qNXSRimItaLCmU0wZYA9ExB7NabCm98CgmlkgnY0gmlwhKwSavuKbXVsdGlhZGRyc5YACASsEmr7Bp7RAAoErBJq-wae0t0DgnJzhQADAQAAiXNlY3AyNTZrMaEColr-TLEdSGe1ehqAIMaWqT_XYNHPa9OASWGGpRCNoGmDdGNwgp7Rg3VkcIKe04V3YWt1MhI"}'
2026-03-18 04:49:50.546 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:49:50.546 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40656/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.107.63/tcp/16952/p2p/16Uiu2HAmMmbSVEQZQTjFUwU9qAPpbNgN5faLkpevhZzhYayvbQ3P"]'
2026-03-18 04:49:50.582 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:50.583 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:49:50.583 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:49:50.585 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:49:50.585 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.181.72
2026-03-18 04:49:50.586 DEBUG [src.node.docker_mananger] Generated ports ['4683', '4684', '4685', '4686', '4687']
2026-03-18 04:49:50.586 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:49:50.587 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:49:50.587 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:49:50.587 DEBUG [src.node.docker_mananger] docker run -i -t -p 4683:4683 -p 4684:4684 -p 4685:4685 -p 4686:4686 -p 4687:4687 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=4685 --rest-port=4683 --tcp-port=4684 --discv5-udp-port=4686 --rest-address=0.0.0.0 --nat=extip:172.18.181.72 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=4c81cc0b5be316faf2bb9f2bf9725ccb298f28d47d6abc235ce18cc085f6db0d --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=4687 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-18 04:49:50.797 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.181.72 waku c0a7eff65992335aaac50286349b371b6426b9b94a25ad206d4186ad2b664399
2026-03-18 04:49:50.839 DEBUG [src.node.docker_mananger] Container started with ID c0a7eff65992. Setting up logs at ./log/docker/node8_2026-03-18_04-49-41__bbabd094-9f5d-45fe-bd27-9ab645f02be2__wakuorg_nwaku:latest.log
2026-03-18 04:49:50.839 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 4683
2026-03-18 04:49:50.839 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:49:51.840 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4683/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:51.843 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:49:51.843 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:49:51.843 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4683/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:49:51.846 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.181.72/tcp/4684/p2p/16Uiu2HAmSvq69Tq6Qq1CX8spSqojrNSwmvjH4rE8cZ6eX9PSgSQd","/ip4/172.18.181.72/tcp/4685/ws/p2p/16Uiu2HAmSvq69Tq6Qq1CX8spSqojrNSwmvjH4rE8cZ6eX9PSgSQd"],"enrUri":"enr:-L24QF9R4CQYjNLmpQJkXEagqKPyIgk8-drSHAzMTE_y38UeSBIdM1ZVsnlkcuwxonzl66qGgmmHNM4jTGnZG7ZOuZkCgmlkgnY0gmlwhKwStUiKbXVsdGlhZGRyc5YACASsErVIBhJMAAoErBK1SAYSTd0DgnJzhQADAQAAiXNlY3AyNTZrMaED1BJwONu5-WU4fmzNtj9UX4Mzas_dS9O87l6pTGRRCEaDdGNwghJMg3VkcIISToV3YWt1MhI"}'
2026-03-18 04:49:51.846 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:49:51.846 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:4683/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.106.251/tcp/40657/p2p/16Uiu2HAm6MSW54pt6uBdcazZCvk7fdGuYZFrx1wPRkq1petVZ3Ha"]'
2026-03-18 04:49:51.879 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:51.881 DEBUG [src.steps.store] Relaying message
2026-03-18 04:49:51.882 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48861/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:49:51.890 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:51.891 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-18 04:49:52.091 DEBUG [src.steps.store] Relaying message
2026-03-18 04:49:52.092 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48861/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:49:52.096 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:52.096 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-18 04:49:52.297 DEBUG [src.steps.store] Relaying message
2026-03-18 04:49:52.297 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48861/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:49:52.302 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:52.302 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-18 04:49:52.502 DEBUG [src.steps.store] Relaying message
2026-03-18 04:49:52.503 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48861/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:49:52.507 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:52.507 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-18 04:49:52.708 DEBUG [src.steps.store] Relaying message
2026-03-18 04:49:52.708 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48861/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:49:52.713 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:52.713 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-18 04:49:52.914 DEBUG [src.steps.store] Relaying message
2026-03-18 04:49:52.914 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48861/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:49:52.919 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:52.919 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-18 04:49:53.120 DEBUG [src.steps.store] Relaying message
2026-03-18 04:49:53.120 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48861/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:49:53.124 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:53.125 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-18 04:49:53.325 DEBUG [src.steps.store] Relaying message
2026-03-18 04:49:53.326 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48861/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:49:53.330 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:53.330 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-18 04:49:53.531 DEBUG [src.steps.store] Relaying message
2026-03-18 04:49:53.531 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48861/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:49:53.536 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:53.536 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-18 04:49:53.736 DEBUG [src.steps.store] Relaying message
2026-03-18 04:49:53.737 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48861/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:49:53.741 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:49:53.741 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-18 04:49:53.942 DEBUG [src.libs.common] Sleeping for 90 seconds
2026-03-18 04:51:23.942 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-18 04:51:23.943 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4683/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:51:23.947 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0xa523f158b8949c7341304d3e8c4167e0c3791e0f81692a0e53f8df418952bd15"},{"messageHash":"0xb18f72ddf8e64fe66444d23b77e0c9e1027b40f40ab40df9359d4f730ccc273d"},{"messageHash":"0x8f441e174eb813831219f19567159a1f2dfde0b0efd896d302952d6221651b8e"},{"messageHash":"0x583d885da43fc879cfd742e50ede64c3224e8aa1490e99b78ffc9de16ecb0cfc"},{"messageHash":"0x31e5f34ee3a1152761165109b450f683e16b507b832e7a0eae7085572394e0ee"},{"messageHash":"0xfc62dad416efd4479add004500038890d9cddad46a0753c89922954bb4c43044"},{"messageHash":"0xe6e84b9d31dba7140ba2260220cea912c0b8787aaff843c2356c41a1d3796be3"},{"messageHash":"0xd4132cf383b1903fd8a70104f3edb18cf3c26ff101a358e94478eb156407165f"},{"messageHash":"0xeaf17be77ad7bcb9d8367fd95169edfd12a2d446ae72af58b94bf3ae941c341a"},{"messageHash":"0x8213db3b382d41fbd284adb6c0445668d9d32f958c5ad7e08a33a5f354f7b7fe"}]}'
2026-03-18 04:51:23.947 DEBUG [src.steps.store] messages length is 10
2026-03-18 04:51:23.950 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-18 04:51:23.951 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-18 04:51:23.951 DEBUG [src.node.waku_node] Stopping container with id 63ab676e0ddc
2026-03-18 04:51:24.544 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:51:24.545 DEBUG [src.node.waku_node] Stopping container with id 674ff9c1063a
2026-03-18 04:51:25.135 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:51:25.138 DEBUG [src.node.waku_node] Stopping container with id 38c94748a3c1
2026-03-18 04:51:25.361 ERROR [src.node.docker_mananger] Max retries reached for container 63ab676e0ddc. Exiting log stream.
2026-03-18 04:51:25.744 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:51:25.744 DEBUG [src.node.waku_node] Stopping container with id c153586597e1
2026-03-18 04:51:25.954 ERROR [src.node.docker_mananger] Max retries reached for container 674ff9c1063a. Exiting log stream.
2026-03-18 04:51:26.307 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:51:26.307 DEBUG [src.node.waku_node] Stopping container with id f22272016117
2026-03-18 04:51:26.562 ERROR [src.node.docker_mananger] Max retries reached for container 38c94748a3c1. Exiting log stream.
2026-03-18 04:51:26.883 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:51:26.884 DEBUG [src.node.waku_node] Stopping container with id 1fe0f1cc4a41
2026-03-18 04:51:27.121 ERROR [src.node.docker_mananger] Max retries reached for container c153586597e1. Exiting log stream.
2026-03-18 04:51:27.484 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:51:27.487 DEBUG [src.node.waku_node] Stopping container with id 9f40f809d825
2026-03-18 04:51:27.697 ERROR [src.node.docker_mananger] Max retries reached for container f22272016117. Exiting log stream.
2026-03-18 04:51:28.049 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:51:28.049 DEBUG [src.node.waku_node] Stopping container with id c0a7eff65992
2026-03-18 04:51:28.298 ERROR [src.node.docker_mananger] Max retries reached for container 1fe0f1cc4a41. Exiting log stream.
2026-03-18 04:51:28.605 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:51:28.608 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-18 04:51:28.644 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:51:28.700 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:51:28.748 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:51:28.793 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:51:28.838 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:51:28.882 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:51:28.930 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:51:28.959 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:51:28.961 ERROR [src.node.docker_mananger] Max retries reached for container 9f40f809d825. Exiting log stream.