2026-03-05 15:25:13 +00:00

270 lines
49 KiB
Plaintext

2026-03-05 15:15:24.258 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-05 15:15:24.258 DEBUG [tests.conftest] Running test: test_store_sync_long_chain with id: 2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a
2026-03-05 15:15:24.258 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-05 15:15:24.267 DEBUG [src.node.docker_mananger] Docker client initialized with image harbor.status.im/wakuorg/nwaku:v0.38.0-beta
2026-03-05 15:15:24.267 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:24.275 DEBUG [src.node.docker_mananger] Docker client initialized with image harbor.status.im/wakuorg/nwaku:v0.38.0-beta
2026-03-05 15:15:24.275 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:24.284 DEBUG [src.node.docker_mananger] Docker client initialized with image harbor.status.im/wakuorg/nwaku:v0.38.0-beta
2026-03-05 15:15:24.284 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:24.284 DEBUG [src.steps.store] Running fixture setup: store_setup
2026-03-05 15:15:24.285 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 15:15:24.286 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 15:15:24.287 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 15:15:24.288 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.212.48
2026-03-05 15:15:24.288 DEBUG [src.node.docker_mananger] Generated ports ['40829', '40830', '40831', '40832', '40833']
2026-03-05 15:15:24.288 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 15:15:24.288 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 15:15:24.288 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 15:15:24.288 DEBUG [src.node.docker_mananger] docker run -i -t -p 40829:40829 -p 40830:40830 -p 40831:40831 -p 40832:40832 -p 40833:40833 harbor.status.im/wakuorg/nwaku:v0.38.0-beta --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=40831 --rest-port=40829 --tcp-port=40830 --discv5-udp-port=40832 --rest-address=0.0.0.0 --nat=extip:172.18.212.48 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9bf9fcf83c047fabe4b83fa8da6ad9cf84fcc6e4bbbe7decfebc60bfbb4d9b01 --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=40833 --metrics-logging=true --store-sync-relay-jitter=0 --relay=true --dns-discovery=false
2026-03-05 15:15:24.458 ERROR [src.node.docker_mananger] Max retries reached for container 08e7f8fbf670. Exiting log stream.
2026-03-05 15:15:24.532 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.212.48 waku 5f5e3f5f1ad7a6789fe8f80637d195d58a5bea62cb6ce99ebd94263365972272
2026-03-05 15:15:24.567 DEBUG [src.node.docker_mananger] Container started with ID 5f5e3f5f1ad7. Setting up logs at ./log/docker/node1_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:24.568 DEBUG [src.node.waku_node] Started container from image harbor.status.im/wakuorg/nwaku:v0.38.0-beta. REST: 40829
2026-03-05 15:15:24.568 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 15:15:25.053 ERROR [src.node.docker_mananger] Max retries reached for container 368934cbf1cd. Exiting log stream.
2026-03-05 15:15:25.569 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:40829/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:25.572 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 15:15:25.572 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 15:15:25.572 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:40829/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:25.575 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.212.48/tcp/40830/p2p/16Uiu2HAm9iTY8scHwMnJWsnEFsJiuNsA8h7Ru9Y1KcYtCy7wohG9","/ip4/172.18.212.48/tcp/40831/ws/p2p/16Uiu2HAm9iTY8scHwMnJWsnEFsJiuNsA8h7Ru9Y1KcYtCy7wohG9"],"enrUri":"enr:-L24QBwtHhQ3hhXTfbwoL4ZAsowTB7a0nDQobWCXbwbNYlRxFw13kpZVU9DM3eFwF1mCbsFMTdzCZizGcnZvOlaoe4cCgmlkgnY0gmlwhKwS1DCKbXVsdGlhZGRyc5YACASsEtQwBp9-AAoErBLUMAaff90DgnJzhQADAQAAiXNlY3AyNTZrMaEC1FBsDsJw9ZX0qvaOFTuaeL0SVh6KANihkQTrdyNp7d6DdGNwgp9-g3VkcIKfgIV3YWt1MhM"}'
2026-03-05 15:15:25.575 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 15:15:25.576 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40829/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-05 15:15:25.595 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:25.603 DEBUG [src.node.docker_mananger] Docker client initialized with image harbor.status.im/wakuorg/nwaku:v0.38.0-beta
2026-03-05 15:15:25.603 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:25.612 DEBUG [src.node.docker_mananger] Docker client initialized with image harbor.status.im/wakuorg/nwaku:v0.38.0-beta
2026-03-05 15:15:25.612 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:25.621 DEBUG [src.node.docker_mananger] Docker client initialized with image harbor.status.im/wakuorg/nwaku:v0.38.0-beta
2026-03-05 15:15:25.622 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node6_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:25.630 DEBUG [src.node.docker_mananger] Docker client initialized with image harbor.status.im/wakuorg/nwaku:v0.38.0-beta
2026-03-05 15:15:25.631 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node7_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:25.639 DEBUG [src.node.docker_mananger] Docker client initialized with image harbor.status.im/wakuorg/nwaku:v0.38.0-beta
2026-03-05 15:15:25.639 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node8_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:25.639 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 15:15:25.640 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 15:15:25.642 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 15:15:25.642 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.244.125
2026-03-05 15:15:25.642 DEBUG [src.node.docker_mananger] Generated ports ['62045', '62046', '62047', '62048', '62049']
2026-03-05 15:15:25.642 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 15:15:25.642 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 15:15:25.642 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 15:15:25.643 DEBUG [src.node.docker_mananger] docker run -i -t -p 62045:62045 -p 62046:62046 -p 62047:62047 -p 62048:62048 -p 62049:62049 harbor.status.im/wakuorg/nwaku:v0.38.0-beta --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=62047 --rest-port=62045 --tcp-port=62046 --discv5-udp-port=62048 --rest-address=0.0.0.0 --nat=extip:172.18.244.125 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=3ef8bf0eef935c5ca20efbeea3575f715ed6bd4c8ab6a5dca8eb3ee89ea53a55 --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=62049 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-05 15:15:25.873 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.244.125 waku 0a4d6152d1c56ff4779f6009fab6a3603abd4c9e0018821117011423d811f0f6
2026-03-05 15:15:25.929 DEBUG [src.node.docker_mananger] Container started with ID 0a4d6152d1c5. Setting up logs at ./log/docker/node2_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:25.930 DEBUG [src.node.waku_node] Started container from image harbor.status.im/wakuorg/nwaku:v0.38.0-beta. REST: 62045
2026-03-05 15:15:25.931 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 15:15:26.933 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:62045/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:26.936 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 15:15:26.936 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 15:15:26.936 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:62045/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:26.939 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.244.125/tcp/62046/p2p/16Uiu2HAmBkAzk4rccpp3AcyvHjJ6MWfofsHrs7YCQLGN34h5Shb1","/ip4/172.18.244.125/tcp/62047/ws/p2p/16Uiu2HAmBkAzk4rccpp3AcyvHjJ6MWfofsHrs7YCQLGN34h5Shb1"],"enrUri":"enr:-L24QBoGMpO3Qzt2KzZSqJRpBmqV2oNglAsQ8CR2hdxR2-vGX5cZiTMl_eCmI_UlVPKO8UZiMnXsd8Kf0vFCKpTKFaQCgmlkgnY0gmlwhKwS9H2KbXVsdGlhZGRyc5YACASsEvR9BvJeAAoErBL0fQbyX90DgnJzhQADAQAAiXNlY3AyNTZrMaEC8nhBGjb5svbSVrfg-8mS1m06uYG8T9paUKJXxq5DDzyDdGNwgvJeg3VkcILyYIV3YWt1MhI"}'
2026-03-05 15:15:26.939 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 15:15:26.940 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:62045/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.212.48/tcp/40830/p2p/16Uiu2HAm9iTY8scHwMnJWsnEFsJiuNsA8h7Ru9Y1KcYtCy7wohG9"]'
2026-03-05 15:15:26.976 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:26.978 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 15:15:26.979 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 15:15:26.985 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 15:15:26.985 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.45.205
2026-03-05 15:15:26.988 DEBUG [src.node.docker_mananger] Generated ports ['34575', '34576', '34577', '34578', '34579']
2026-03-05 15:15:26.988 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 15:15:26.989 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 15:15:26.989 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 15:15:26.990 DEBUG [src.node.docker_mananger] docker run -i -t -p 34575:34575 -p 34576:34576 -p 34577:34577 -p 34578:34578 -p 34579:34579 harbor.status.im/wakuorg/nwaku:v0.38.0-beta --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=34577 --rest-port=34575 --tcp-port=34576 --discv5-udp-port=34578 --rest-address=0.0.0.0 --nat=extip:172.18.45.205 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=b81910b2c46aae4abfe7da2f90ab5acfeeeda7de0392795bacfc8b897ad0c4cb --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=34579 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-05 15:15:27.212 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.45.205 waku 9864a495a8c5f7e87f2d019c282a6515bd66dbb240ed1711dcdfd8d9ca5f7c59
2026-03-05 15:15:27.251 DEBUG [src.node.docker_mananger] Container started with ID 9864a495a8c5. Setting up logs at ./log/docker/node3_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:27.251 DEBUG [src.node.waku_node] Started container from image harbor.status.im/wakuorg/nwaku:v0.38.0-beta. REST: 34575
2026-03-05 15:15:27.252 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 15:15:28.255 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:34575/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:28.257 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 15:15:28.258 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 15:15:28.258 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:34575/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:28.261 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.45.205/tcp/34576/p2p/16Uiu2HAm5AcGkuUtLMcBwAfbvh7VRh4BbdcouwCqHzmZPfgDqLfS","/ip4/172.18.45.205/tcp/34577/ws/p2p/16Uiu2HAm5AcGkuUtLMcBwAfbvh7VRh4BbdcouwCqHzmZPfgDqLfS"],"enrUri":"enr:-L24QLr__ToMaOmjbcbJvg2lTC2XDSRaH9ThXrumKPTriaq6OMwbU0QRczSGCUB6HHiw1FFffNe6smeflV_QHsj1PGYCgmlkgnY0gmlwhKwSLc2KbXVsdGlhZGRyc5YACASsEi3NBocQAAoErBItzQaHEd0DgnJzhQADAQAAiXNlY3AyNTZrMaECkLj3DDBQ8Cqhl8o2oaYvfeG7en0chSeNpnT0BDhhAoGDdGNwgocQg3VkcIKHEoV3YWt1MhI"}'
2026-03-05 15:15:28.261 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 15:15:28.261 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:34575/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.244.125/tcp/62046/p2p/16Uiu2HAmBkAzk4rccpp3AcyvHjJ6MWfofsHrs7YCQLGN34h5Shb1"]'
2026-03-05 15:15:28.297 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:28.297 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 15:15:28.299 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 15:15:28.304 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 15:15:28.307 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.208.157
2026-03-05 15:15:28.308 DEBUG [src.node.docker_mananger] Generated ports ['31493', '31494', '31495', '31496', '31497']
2026-03-05 15:15:28.308 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 15:15:28.309 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 15:15:28.309 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 15:15:28.309 DEBUG [src.node.docker_mananger] docker run -i -t -p 31493:31493 -p 31494:31494 -p 31495:31495 -p 31496:31496 -p 31497:31497 harbor.status.im/wakuorg/nwaku:v0.38.0-beta --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=31495 --rest-port=31493 --tcp-port=31494 --discv5-udp-port=31496 --rest-address=0.0.0.0 --nat=extip:172.18.208.157 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=31dcfbf6554ab7752691e6f8ddc5fddf9f315bca9de5ab6d86f5cfba0bea23e5 --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=31497 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-05 15:15:28.539 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.208.157 waku 403ec1a3679348562fd96086ed42a1f77535483c04713c82a394ac9007ad530c
2026-03-05 15:15:28.581 DEBUG [src.node.docker_mananger] Container started with ID 403ec1a36793. Setting up logs at ./log/docker/node4_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:28.582 DEBUG [src.node.waku_node] Started container from image harbor.status.im/wakuorg/nwaku:v0.38.0-beta. REST: 31493
2026-03-05 15:15:28.584 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 15:15:29.585 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:31493/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:29.588 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 15:15:29.589 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 15:15:29.589 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:31493/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:29.592 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.208.157/tcp/31494/p2p/16Uiu2HAmFfZhXCajMAeNHz8wzBvQ6jrTST9zxHvcF2jw9s5K1wLQ","/ip4/172.18.208.157/tcp/31495/ws/p2p/16Uiu2HAmFfZhXCajMAeNHz8wzBvQ6jrTST9zxHvcF2jw9s5K1wLQ"],"enrUri":"enr:-L24QMcZdps4rNZdGKljaQfw9ngFUfvZTjJTnOtsLJ6Upoo_B1NWWpX6OnMfJ0phs2sx2mzKRTV60IgglF9dcQEoy7ICgmlkgnY0gmlwhKwS0J2KbXVsdGlhZGRyc5YACASsEtCdBnsGAAoErBLQnQZ7B90DgnJzhQADAQAAiXNlY3AyNTZrMaEDLLjEtppoab5FyJe4boH_IOaj-oOuXeAUgbig1eWqNh2DdGNwgnsGg3VkcIJ7CIV3YWt1MhI"}'
2026-03-05 15:15:29.593 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 15:15:29.593 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:31493/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.45.205/tcp/34576/p2p/16Uiu2HAm5AcGkuUtLMcBwAfbvh7VRh4BbdcouwCqHzmZPfgDqLfS"]'
2026-03-05 15:15:29.633 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:29.635 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 15:15:29.636 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 15:15:29.641 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 15:15:29.642 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.10.204
2026-03-05 15:15:29.642 DEBUG [src.node.docker_mananger] Generated ports ['1608', '1609', '1610', '1611', '1612']
2026-03-05 15:15:29.642 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 15:15:29.642 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 15:15:29.643 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 15:15:29.643 DEBUG [src.node.docker_mananger] docker run -i -t -p 1608:1608 -p 1609:1609 -p 1610:1610 -p 1611:1611 -p 1612:1612 harbor.status.im/wakuorg/nwaku:v0.38.0-beta --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=1610 --rest-port=1608 --tcp-port=1609 --discv5-udp-port=1611 --rest-address=0.0.0.0 --nat=extip:172.18.10.204 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=b2c38fd2868fb7e9eb00bfc6c90ffe3c3d4b8e20330b7fbb0ab05d44ee1347ae --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=1612 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-05 15:15:29.886 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.10.204 waku 8249aebc0d729a61231f92438ac01cb3465f0e24e7d882693f8d82ac681cdf5c
2026-03-05 15:15:29.926 DEBUG [src.node.docker_mananger] Container started with ID 8249aebc0d72. Setting up logs at ./log/docker/node5_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:29.926 DEBUG [src.node.waku_node] Started container from image harbor.status.im/wakuorg/nwaku:v0.38.0-beta. REST: 1608
2026-03-05 15:15:29.927 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 15:15:30.927 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1608/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:30.930 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 15:15:30.931 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 15:15:30.931 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1608/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:30.934 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.10.204/tcp/1609/p2p/16Uiu2HAm7dthzYzs4bkmT28CdPwuu6eTqt2sJcJzFuqBDKWb13yn","/ip4/172.18.10.204/tcp/1610/ws/p2p/16Uiu2HAm7dthzYzs4bkmT28CdPwuu6eTqt2sJcJzFuqBDKWb13yn"],"enrUri":"enr:-L24QJ-A3vWZGWFiWFhWFBWlEd1u3iFCq-3AqKdJq3XIfH_ZRNjmCx82GMy2Y16H-aoMuqb39s8gG2asW1WW5Gh2RMcCgmlkgnY0gmlwhKwSCsyKbXVsdGlhZGRyc5YACASsEgrMBgZJAAoErBIKzAYGSt0DgnJzhQADAQAAiXNlY3AyNTZrMaECtW2anBi9LFhEN-E94U_Wzplr9wq-87hjo1ulY9Ks_uWDdGNwggZJg3VkcIIGS4V3YWt1MhI"}'
2026-03-05 15:15:30.934 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 15:15:30.935 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:1608/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.208.157/tcp/31494/p2p/16Uiu2HAmFfZhXCajMAeNHz8wzBvQ6jrTST9zxHvcF2jw9s5K1wLQ"]'
2026-03-05 15:15:30.968 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:30.969 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 15:15:30.972 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 15:15:30.976 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 15:15:30.977 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.101.24
2026-03-05 15:15:30.977 DEBUG [src.node.docker_mananger] Generated ports ['34904', '34905', '34906', '34907', '34908']
2026-03-05 15:15:30.977 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 15:15:30.978 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 15:15:30.978 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 15:15:30.978 DEBUG [src.node.docker_mananger] docker run -i -t -p 34904:34904 -p 34905:34905 -p 34906:34906 -p 34907:34907 -p 34908:34908 harbor.status.im/wakuorg/nwaku:v0.38.0-beta --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=34906 --rest-port=34904 --tcp-port=34905 --discv5-udp-port=34907 --rest-address=0.0.0.0 --nat=extip:172.18.101.24 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=4aa68dd2cf0ff4bc3aba071bd9ff33e8673a9aae2dc5cc3d3eecdaf0f0bfb2a8 --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=34908 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-05 15:15:31.212 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.101.24 waku fe4ef7ec568d864b225fe36d4a4216d19e871709e5d272a9a19f71655d62d8ff
2026-03-05 15:15:31.254 DEBUG [src.node.docker_mananger] Container started with ID fe4ef7ec568d. Setting up logs at ./log/docker/node6_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:31.254 DEBUG [src.node.waku_node] Started container from image harbor.status.im/wakuorg/nwaku:v0.38.0-beta. REST: 34904
2026-03-05 15:15:31.255 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 15:15:32.255 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:34904/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:32.258 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 15:15:32.258 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 15:15:32.259 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:34904/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:32.261 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.101.24/tcp/34905/p2p/16Uiu2HAkwuRQNBiQrBaboeYU9aRTC4BkCBJjhj7yYJuAaAF5J4NY","/ip4/172.18.101.24/tcp/34906/ws/p2p/16Uiu2HAkwuRQNBiQrBaboeYU9aRTC4BkCBJjhj7yYJuAaAF5J4NY"],"enrUri":"enr:-L24QKJ_MhresorIHkCYcw96kwLdrF4J1PbPS1WdRt-9lJfxG9rn8HrrayDeoMwziluBAi4zBCAzf6OegU9QbYoOGYoCgmlkgnY0gmlwhKwSZRiKbXVsdGlhZGRyc5YACASsEmUYBohZAAoErBJlGAaIWt0DgnJzhQADAQAAiXNlY3AyNTZrMaECJNMnrY1u7ftUlrNAzgs5DBtGH-lkpdEJHRyVxwMHq7WDdGNwgohZg3VkcIKIW4V3YWt1MhI"}'
2026-03-05 15:15:32.262 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 15:15:32.262 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:34904/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.10.204/tcp/1609/p2p/16Uiu2HAm7dthzYzs4bkmT28CdPwuu6eTqt2sJcJzFuqBDKWb13yn"]'
2026-03-05 15:15:32.301 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:32.302 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 15:15:32.305 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 15:15:32.312 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 15:15:32.314 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.156.125
2026-03-05 15:15:32.314 DEBUG [src.node.docker_mananger] Generated ports ['16772', '16773', '16774', '16775', '16776']
2026-03-05 15:15:32.315 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 15:15:32.315 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 15:15:32.315 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 15:15:32.316 DEBUG [src.node.docker_mananger] docker run -i -t -p 16772:16772 -p 16773:16773 -p 16774:16774 -p 16775:16775 -p 16776:16776 harbor.status.im/wakuorg/nwaku:v0.38.0-beta --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=16774 --rest-port=16772 --tcp-port=16773 --discv5-udp-port=16775 --rest-address=0.0.0.0 --nat=extip:172.18.156.125 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=d71eceaed72369bdc8a15c4b26bd82bf2b911cf6fbd8f3331ab88f42bb27c1ec --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=16776 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-05 15:15:32.560 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.156.125 waku bd7500e80f151323bc8270b1e8dd4f3330eb1e5c6f43edf37631a88f39bb56d3
2026-03-05 15:15:32.603 DEBUG [src.node.docker_mananger] Container started with ID bd7500e80f15. Setting up logs at ./log/docker/node7_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:32.604 DEBUG [src.node.waku_node] Started container from image harbor.status.im/wakuorg/nwaku:v0.38.0-beta. REST: 16772
2026-03-05 15:15:32.604 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 15:15:33.605 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16772/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:33.609 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 15:15:33.609 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 15:15:33.609 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16772/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:33.612 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.156.125/tcp/16773/p2p/16Uiu2HAmLx2JSJQRRVoEqE85cDZ6N77C9iBxEiutARaPpPWEaCCF","/ip4/172.18.156.125/tcp/16774/ws/p2p/16Uiu2HAmLx2JSJQRRVoEqE85cDZ6N77C9iBxEiutARaPpPWEaCCF"],"enrUri":"enr:-L24QHHa76v82l9tZ10zCfq_pdQXgs17wr25ADbUynOW2zTHKN61xeyoLCJhvcGAs4_3FPG6e1MqU_Bos9fHCB6324wCgmlkgnY0gmlwhKwSnH2KbXVsdGlhZGRyc5YACASsEpx9BkGFAAoErBKcfQZBht0DgnJzhQADAQAAiXNlY3AyNTZrMaEDezqTMFWkigG00RNhNaWxsVfmLZkui1YKoEbJs2B1DjCDdGNwgkGFg3VkcIJBh4V3YWt1MhI"}'
2026-03-05 15:15:33.612 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 15:15:33.613 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16772/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.101.24/tcp/34905/p2p/16Uiu2HAkwuRQNBiQrBaboeYU9aRTC4BkCBJjhj7yYJuAaAF5J4NY"]'
2026-03-05 15:15:33.649 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:33.649 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 15:15:33.650 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 15:15:33.656 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 15:15:33.658 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.140.66
2026-03-05 15:15:33.658 DEBUG [src.node.docker_mananger] Generated ports ['28765', '28766', '28767', '28768', '28769']
2026-03-05 15:15:33.658 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 15:15:33.658 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 15:15:33.659 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 15:15:33.659 DEBUG [src.node.docker_mananger] docker run -i -t -p 28765:28765 -p 28766:28766 -p 28767:28767 -p 28768:28768 -p 28769:28769 harbor.status.im/wakuorg/nwaku:v0.38.0-beta --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=28767 --rest-port=28765 --tcp-port=28766 --discv5-udp-port=28768 --rest-address=0.0.0.0 --nat=extip:172.18.140.66 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9ad63c49cbfba6899917037fcccc17eb4e7a0c1b418faaecbe8d140ccf9bf12d --store-sync=true --store=true --store-sync-range=120 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=28769 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false
2026-03-05 15:15:33.897 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.140.66 waku c77862073113b75f5207d7dbb2e977ca99cf4b7856d0dd6e6091b8efae829a3d
2026-03-05 15:15:33.943 DEBUG [src.node.docker_mananger] Container started with ID c77862073113. Setting up logs at ./log/docker/node8_2026-03-05_15-15-24__6dc1fbe1-2432-4d0a-bf61-6e2ac3b68e2a__harbor.status.im_wakuorg_nwaku:v0.38.0-beta.log
2026-03-05 15:15:33.944 DEBUG [src.node.waku_node] Started container from image harbor.status.im/wakuorg/nwaku:v0.38.0-beta. REST: 28765
2026-03-05 15:15:33.946 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 15:15:34.947 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:28765/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:34.950 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 15:15:34.950 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 15:15:34.951 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:28765/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:15:34.953 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.140.66/tcp/28766/p2p/16Uiu2HAm1whnXHuUFG7b4y5FM6MP842qrevYBaKZkfDYtzdnDJD8","/ip4/172.18.140.66/tcp/28767/ws/p2p/16Uiu2HAm1whnXHuUFG7b4y5FM6MP842qrevYBaKZkfDYtzdnDJD8"],"enrUri":"enr:-L24QLQ85q83PXuFSM7QlsIcOMp3XKGy7R_j_X9F0W4KzdfjJQ8y1WnI6I0u2MBqNFWOtNkLdF8GGms-AIOhM01BixcCgmlkgnY0gmlwhKwSjEKKbXVsdGlhZGRyc5YACASsEoxCBnBeAAoErBKMQgZwX90DgnJzhQADAQAAiXNlY3AyNTZrMaECYNeWZ9KNiM27tpgU01EhD_CQ45bdM6oH5wEkkQhL1RODdGNwgnBeg3VkcIJwYIV3YWt1MhI"}'
2026-03-05 15:15:34.953 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 15:15:34.954 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:28765/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.156.125/tcp/16773/p2p/16Uiu2HAmLx2JSJQRRVoEqE85cDZ6N77C9iBxEiutARaPpPWEaCCF"]'
2026-03-05 15:15:34.994 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:34.994 DEBUG [src.steps.store] Relaying message
2026-03-05 15:15:34.999 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40829/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 15:15:35.005 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:35.005 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-05 15:15:35.206 DEBUG [src.steps.store] Relaying message
2026-03-05 15:15:35.206 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40829/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 15:15:35.210 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:35.211 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-05 15:15:35.411 DEBUG [src.steps.store] Relaying message
2026-03-05 15:15:35.412 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40829/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 15:15:35.416 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:35.416 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-05 15:15:35.617 DEBUG [src.steps.store] Relaying message
2026-03-05 15:15:35.617 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40829/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 15:15:35.621 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:35.622 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-05 15:15:35.822 DEBUG [src.steps.store] Relaying message
2026-03-05 15:15:35.823 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40829/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 15:15:35.828 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:35.828 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-05 15:15:36.029 DEBUG [src.steps.store] Relaying message
2026-03-05 15:15:36.029 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40829/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 15:15:36.034 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:36.034 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-05 15:15:36.235 DEBUG [src.steps.store] Relaying message
2026-03-05 15:15:36.235 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40829/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 15:15:36.240 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:36.240 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-05 15:15:36.441 DEBUG [src.steps.store] Relaying message
2026-03-05 15:15:36.441 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40829/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 15:15:36.445 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:36.446 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-05 15:15:36.646 DEBUG [src.steps.store] Relaying message
2026-03-05 15:15:36.647 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40829/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 15:15:36.651 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:36.651 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-05 15:15:36.852 DEBUG [src.steps.store] Relaying message
2026-03-05 15:15:36.852 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:40829/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 15:15:36.856 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 15:15:36.856 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-05 15:15:37.057 DEBUG [src.libs.common] Sleeping for 90 seconds
2026-03-05 15:17:07.058 DEBUG [src.steps.store] Checking that peer harbor.status.im/wakuorg/nwaku:v0.38.0-beta can find the stored messages
2026-03-05 15:17:07.058 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:28765/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-05 15:17:07.061 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0xd5817f533c8360ed6b0cef699651d1715c8cd26b93cca3cf72df371ac9723056"},{"messageHash":"0xb82a340a4033310fc69b5ea057500f2809cede2fdba2d4ea6b9aeeb056ac3392"},{"messageHash":"0x9ea4ae48158ebd3bb5a3897609f3c16bf0a689e6d146ae9fc567a3343a33fb1f"},{"messageHash":"0x637ad49f470987028ac93fb55e622217fce42fd3262fd67b8b9b1dd521efa04a"},{"messageHash":"0x95fe2e9369c27d8b2f4a559571f74b44de7d58ac6accab3b46076824f2c549fb"},{"messageHash":"0x256abc256f9800802dbeaa7104af0ff1eec1c979044b3749cb8dad972c95b030"},{"messageHash":"0x67197a5d437c2190105c2cae3b8bc5424801328d43047a4e28f5a350713c05c2"},{"messageHash":"0x428d73bd436662a19c6b7f95612da6e548e37411e11a27832dd0591130c9700d"},{"messageHash":"0x4b4f997e608ef19ffaaeaad378d70b48246ebb89c4b51ca60db7277933460082"},{"messageHash":"0x692af6f2bf826877971ced492683adf9dd9676def1b22541ce69c5a364d70c8d"}]}'
2026-03-05 15:17:07.062 DEBUG [src.steps.store] messages length is 10
2026-03-05 15:17:07.064 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-05 15:17:07.065 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-05 15:17:07.066 DEBUG [src.node.waku_node] Stopping container with id 5f5e3f5f1ad7
2026-03-05 15:17:07.659 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 15:17:07.659 DEBUG [src.node.waku_node] Stopping container with id 0a4d6152d1c5
2026-03-05 15:17:08.293 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 15:17:08.295 DEBUG [src.node.waku_node] Stopping container with id 9864a495a8c5
2026-03-05 15:17:08.472 ERROR [src.node.docker_mananger] Max retries reached for container 5f5e3f5f1ad7. Exiting log stream.
2026-03-05 15:17:08.908 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 15:17:08.909 DEBUG [src.node.waku_node] Stopping container with id 403ec1a36793
2026-03-05 15:17:09.110 ERROR [src.node.docker_mananger] Max retries reached for container 0a4d6152d1c5. Exiting log stream.
2026-03-05 15:17:09.500 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 15:17:09.502 DEBUG [src.node.waku_node] Stopping container with id 8249aebc0d72
2026-03-05 15:17:09.889 ERROR [src.node.docker_mananger] Max retries reached for container 9864a495a8c5. Exiting log stream.
2026-03-05 15:17:10.031 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 15:17:10.031 DEBUG [src.node.waku_node] Stopping container with id fe4ef7ec568d
2026-03-05 15:17:10.315 ERROR [src.node.docker_mananger] Max retries reached for container 403ec1a36793. Exiting log stream.
2026-03-05 15:17:10.610 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 15:17:10.611 DEBUG [src.node.waku_node] Stopping container with id bd7500e80f15
2026-03-05 15:17:10.845 ERROR [src.node.docker_mananger] Max retries reached for container 8249aebc0d72. Exiting log stream.
2026-03-05 15:17:11.154 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 15:17:11.154 DEBUG [src.node.waku_node] Stopping container with id c77862073113
2026-03-05 15:17:11.425 ERROR [src.node.docker_mananger] Max retries reached for container fe4ef7ec568d. Exiting log stream.
2026-03-05 15:17:11.726 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 15:17:11.729 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-05 15:17:11.764 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 15:17:11.817 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 15:17:11.908 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 15:17:11.952 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 15:17:11.993 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 15:17:12.038 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 15:17:12.089 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 15:17:12.090 ERROR [src.node.docker_mananger] Max retries reached for container bd7500e80f15. Exiting log stream.
2026-03-05 15:17:12.119 DEBUG [src.node.docker_mananger] No errors found in the waku logs.