185 lines
31 KiB
Plaintext

2026-03-12 04:35:14.413 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-12 04:35:14.413 DEBUG [tests.conftest] Running test: test_retrieve_messages_without_subscribing_via_api with id: 2026-03-12_04-35-14__4fe255b9-3c18-4481-ba5b-3dccee4f9e83
2026-03-12 04:35:14.413 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-12 04:35:14.414 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-12 04:35:14.414 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-12 04:35:14.422 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-12 04:35:14.422 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-12_04-35-14__4fe255b9-3c18-4481-ba5b-3dccee4f9e83__wakuorg_nwaku:latest.log
2026-03-12 04:35:14.422 DEBUG [src.node.waku_node] Starting Node...
2026-03-12 04:35:14.422 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-12 04:35:14.424 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-12 04:35:14.424 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.129.169
2026-03-12 04:35:14.424 DEBUG [src.node.docker_mananger] Generated ports ['22337', '22338', '22339', '22340', '22341']
2026-03-12 04:35:14.424 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-12 04:35:14.424 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-12 04:35:14.424 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-12 04:35:14.425 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-12 04:35:14.425 DEBUG [src.node.waku_node] Using volumes []
2026-03-12 04:35:14.425 DEBUG [src.node.docker_mananger] docker run -i -t -p 22337:22337 -p 22338:22338 -p 22339:22339 -p 22340:22340 -p 22341:22341 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=22339 --rest-port=22337 --tcp-port=22338 --discv5-udp-port=22340 --rest-address=0.0.0.0 --nat=extip:172.18.129.169 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=fbb2c7fbdcafedd9eaa46ed7e3bff508422b8782b77b6816e41ac84ad1603696 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=22341 --metrics-logging=true --relay=true --filter=true
2026-03-12 04:35:14.623 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.129.169 waku f1dcca47d5e7d7a7014fc29bb7fb3a1f21b401d470e8e245f085e83a69d498cf
2026-03-12 04:35:14.661 DEBUG [src.node.docker_mananger] Container started with ID f1dcca47d5e7. Setting up logs at ./log/docker/node1_2026-03-12_04-35-14__4fe255b9-3c18-4481-ba5b-3dccee4f9e83__wakuorg_nwaku:latest.log
2026-03-12 04:35:14.661 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 22337
2026-03-12 04:35:14.663 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-12 04:35:14.811 ERROR [src.node.docker_mananger] Max retries reached for container c63170d43e67. Exiting log stream.
2026-03-12 04:35:15.100 ERROR [src.node.docker_mananger] Max retries reached for container d23f2c2f1d5f. Exiting log stream.
2026-03-12 04:35:15.664 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22337/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:15.667 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:15.767 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22337/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:15.770 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:15.870 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22337/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:15.873 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:15.974 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22337/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:15.976 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:16.076 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22337/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:16.079 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:16.180 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22337/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:16.187 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:16.288 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22337/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:16.296 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-12 04:35:16.296 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-12 04:35:16.296 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22337/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:16.298 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.129.169/tcp/22338/p2p/16Uiu2HAmCYc1nYQ38NTHa8zB8gk11gpHsdjc19ZGdpBvL9M8LzWm","/ip4/172.18.129.169/tcp/22339/ws/p2p/16Uiu2HAmCYc1nYQ38NTHa8zB8gk11gpHsdjc19ZGdpBvL9M8LzWm"],"enrUri":"enr:-L24QJETQT_UzodCR1IIGnXuYDSazeCqaBLNlAS_XVr55mZiPkgwcxtLV0pjlsb-WmWafLPi7wxZRhcUV62AnGKP-osCgmlkgnY0gmlwhKwSgamKbXVsdGlhZGRyc5YACASsEoGpBldCAAoErBKBqQZXQ90DgnJzhQACAQAAiXNlY3AyNTZrMaEC_l1CarJYFjSxtuJLVpOBx44iwfCpttF-eTibsT2eDyqDdGNwgldCg3VkcIJXRIV3YWt1MgU"}'
2026-03-12 04:35:16.299 INFO [src.node.waku_node] REST service is ready !!
2026-03-12 04:35:16.306 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-12 04:35:16.306 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-12_04-35-14__4fe255b9-3c18-4481-ba5b-3dccee4f9e83__wakuorg_nwaku:latest.log
2026-03-12 04:35:16.307 DEBUG [src.node.waku_node] Starting Node...
2026-03-12 04:35:16.307 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-12 04:35:16.308 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-12 04:35:16.308 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.161.202
2026-03-12 04:35:16.308 DEBUG [src.node.docker_mananger] Generated ports ['22104', '22105', '22106', '22107', '22108']
2026-03-12 04:35:16.309 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-12 04:35:16.309 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-12 04:35:16.309 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-12 04:35:16.309 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-12 04:35:16.309 DEBUG [src.node.waku_node] Using volumes []
2026-03-12 04:35:16.309 DEBUG [src.node.docker_mananger] docker run -i -t -p 22104:22104 -p 22105:22105 -p 22106:22106 -p 22107:22107 -p 22108:22108 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=22106 --rest-port=22104 --tcp-port=22105 --discv5-udp-port=22107 --rest-address=0.0.0.0 --nat=extip:172.18.161.202 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=3fdd1c7ddbf7bfee2732b95f7d4778f81befdef07fe9475e4ccb6dad1ed0a16a --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=22108 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QJETQT_UzodCR1IIGnXuYDSazeCqaBLNlAS_XVr55mZiPkgwcxtLV0pjlsb-WmWafLPi7wxZRhcUV62AnGKP-osCgmlkgnY0gmlwhKwSgamKbXVsdGlhZGRyc5YACASsEoGpBldCAAoErBKBqQZXQ90DgnJzhQACAQAAiXNlY3AyNTZrMaEC_l1CarJYFjSxtuJLVpOBx44iwfCpttF-eTibsT2eDyqDdGNwgldCg3VkcIJXRIV3YWt1MgU
2026-03-12 04:35:16.508 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.161.202 waku fadd4d4a6c1c5d97d3593d7c4e5561a226996cad0a5eaf3c55a16e0cf284e72e
2026-03-12 04:35:16.542 DEBUG [src.node.docker_mananger] Container started with ID fadd4d4a6c1c. Setting up logs at ./log/docker/node2_2026-03-12_04-35-14__4fe255b9-3c18-4481-ba5b-3dccee4f9e83__wakuorg_nwaku:latest.log
2026-03-12 04:35:16.542 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 22104
2026-03-12 04:35:16.542 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-12 04:35:17.543 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22104/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:17.546 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:17.646 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22104/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:17.650 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:17.751 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22104/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:17.754 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-12 04:35:17.754 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-12 04:35:17.754 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22104/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:17.757 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.161.202/tcp/22105/p2p/16Uiu2HAmSwjbjLyE72UZmRJu7EdvwTSKenEFB2DPzmHRpMfEkvfd","/ip4/172.18.161.202/tcp/22106/ws/p2p/16Uiu2HAmSwjbjLyE72UZmRJu7EdvwTSKenEFB2DPzmHRpMfEkvfd"],"enrUri":"enr:-L24QJFVjkNFbhSxGcj2BbqX7P8DeDaKo-W3yi3qHk2aYNTVKeU8N0h6Iqjrro72gTXIff_27rzDcZCTVbsDAyIYtLUCgmlkgnY0gmlwhKwSocqKbXVsdGlhZGRyc5YACASsEqHKBlZZAAoErBKhygZWWt0DgnJzhQACAQAAiXNlY3AyNTZrMaED1E3Py0BodsgcxN79lQjdFYQ1yvfN3SKyq-yiFgMUv_yDdGNwglZZg3VkcIJWW4V3YWt1MgE"}'
2026-03-12 04:35:17.757 INFO [src.node.waku_node] REST service is ready !!
2026-03-12 04:35:17.757 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:22104/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.129.169/tcp/22338/p2p/16Uiu2HAmCYc1nYQ38NTHa8zB8gk11gpHsdjc19ZGdpBvL9M8LzWm"]'
2026-03-12 04:35:17.776 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:35:17.777 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:22337/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-12 04:35:17.799 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:35:17.800 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-12 04:35:17.902 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-12 04:35:17.903 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:22337/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:17.905 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773290113984795374,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773290117487635865,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773290117777085378,"ephemeral":false,"proof":""}]'
2026-03-12 04:35:17.916 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-12 04:35:17.917 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-12 04:35:17.917 DEBUG [src.node.waku_node] Stopping container with id f1dcca47d5e7
2026-03-12 04:35:18.469 DEBUG [src.node.waku_node] Container stopped.
2026-03-12 04:35:18.469 DEBUG [src.node.waku_node] Stopping container with id fadd4d4a6c1c
2026-03-12 04:35:19.041 DEBUG [src.node.waku_node] Container stopped.
2026-03-12 04:35:19.044 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-12 04:35:19.081 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-12 04:35:19.108 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-12 04:35:19.109 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-12 04:35:19.110 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-12 04:35:19.110 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-12_04-35-14__4fe255b9-3c18-4481-ba5b-3dccee4f9e83__wakuorg_nwaku:latest.log
2026-03-12 04:35:19.111 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-12_04-35-14__4fe255b9-3c18-4481-ba5b-3dccee4f9e83__wakuorg_nwaku:latest.log
2026-03-12 04:35:19.115 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-12 04:35:19.116 DEBUG [tests.conftest] Running test: test_retrieve_messages_without_subscribing_via_api with id: 2026-03-12_04-35-19__49444a02-ea45-40c8-8320-108aefb411cc
2026-03-12 04:35:19.116 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-12 04:35:19.116 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-12 04:35:19.116 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-12 04:35:19.124 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-12 04:35:19.124 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-12_04-35-19__49444a02-ea45-40c8-8320-108aefb411cc__wakuorg_nwaku:latest.log
2026-03-12 04:35:19.124 DEBUG [src.node.waku_node] Starting Node...
2026-03-12 04:35:19.124 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-12 04:35:19.126 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-12 04:35:19.126 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.180.7
2026-03-12 04:35:19.126 DEBUG [src.node.docker_mananger] Generated ports ['19501', '19502', '19503', '19504', '19505']
2026-03-12 04:35:19.126 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-12 04:35:19.127 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-12 04:35:19.127 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-12 04:35:19.127 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-12 04:35:19.127 DEBUG [src.node.waku_node] Using volumes []
2026-03-12 04:35:19.127 DEBUG [src.node.docker_mananger] docker run -i -t -p 19501:19501 -p 19502:19502 -p 19503:19503 -p 19504:19504 -p 19505:19505 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=19503 --rest-port=19501 --tcp-port=19502 --discv5-udp-port=19504 --rest-address=0.0.0.0 --nat=extip:172.18.180.7 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=d1d37eea6c7bc4c4a3cece23ccacccaebe78ae78dfff522449b8b27da3dfbbd2 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=19505 --metrics-logging=true --relay=true --filter=true
2026-03-12 04:35:19.312 ERROR [src.node.docker_mananger] Max retries reached for container f1dcca47d5e7. Exiting log stream.
2026-03-12 04:35:19.323 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.180.7 waku cbf49b1a2b51c883bdd05709296d9ae90f2eb013f4158e17c3abba4725cc08bd
2026-03-12 04:35:19.356 DEBUG [src.node.docker_mananger] Container started with ID cbf49b1a2b51. Setting up logs at ./log/docker/node1_2026-03-12_04-35-19__49444a02-ea45-40c8-8320-108aefb411cc__wakuorg_nwaku:latest.log
2026-03-12 04:35:19.356 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 19501
2026-03-12 04:35:19.357 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-12 04:35:19.855 ERROR [src.node.docker_mananger] Max retries reached for container fadd4d4a6c1c. Exiting log stream.
2026-03-12 04:35:20.357 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19501/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:20.360 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:20.460 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19501/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:20.497 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:20.597 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19501/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:20.600 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-12 04:35:20.600 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-12 04:35:20.600 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19501/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:20.603 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.180.7/tcp/19502/p2p/16Uiu2HAm5qnuC1pLkb5T2bz9FhntgA7g539mkpP831nD8FopVtUg","/ip4/172.18.180.7/tcp/19503/ws/p2p/16Uiu2HAm5qnuC1pLkb5T2bz9FhntgA7g539mkpP831nD8FopVtUg"],"enrUri":"enr:-L24QBjQ4LsmW2NNUC5XDCkdRRPey43wsoKD15sIhGo5mTANNn3yNkgVb8ZhDQIm5DAAbt5lXvRpdqp4PQn1zwoX6hsCgmlkgnY0gmlwhKwStAeKbXVsdGlhZGRyc5YACASsErQHBkwuAAoErBK0BwZML90DgnJzhQACAQAAiXNlY3AyNTZrMaECmsKiRajpqbh6E-WWgtlpP5kiqIRZy53GxcsAizyh6IGDdGNwgkwug3VkcIJMMIV3YWt1MgU"}'
2026-03-12 04:35:20.603 INFO [src.node.waku_node] REST service is ready !!
2026-03-12 04:35:20.610 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-12 04:35:20.610 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-12_04-35-19__49444a02-ea45-40c8-8320-108aefb411cc__wakuorg_nwaku:latest.log
2026-03-12 04:35:20.611 DEBUG [src.node.waku_node] Starting Node...
2026-03-12 04:35:20.611 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-12 04:35:20.612 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-12 04:35:20.612 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.175.121
2026-03-12 04:35:20.612 DEBUG [src.node.docker_mananger] Generated ports ['5681', '5682', '5683', '5684', '5685']
2026-03-12 04:35:20.613 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-12 04:35:20.613 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-12 04:35:20.613 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-12 04:35:20.613 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-12 04:35:20.613 DEBUG [src.node.waku_node] Using volumes []
2026-03-12 04:35:20.613 DEBUG [src.node.docker_mananger] docker run -i -t -p 5681:5681 -p 5682:5682 -p 5683:5683 -p 5684:5684 -p 5685:5685 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=5683 --rest-port=5681 --tcp-port=5682 --discv5-udp-port=5684 --rest-address=0.0.0.0 --nat=extip:172.18.175.121 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=7ded027bbccaccf1e9ad4f29eb439ab6a5a9fc731db8fbb677b55addb856d431 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=5685 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QBjQ4LsmW2NNUC5XDCkdRRPey43wsoKD15sIhGo5mTANNn3yNkgVb8ZhDQIm5DAAbt5lXvRpdqp4PQn1zwoX6hsCgmlkgnY0gmlwhKwStAeKbXVsdGlhZGRyc5YACASsErQHBkwuAAoErBK0BwZML90DgnJzhQACAQAAiXNlY3AyNTZrMaECmsKiRajpqbh6E-WWgtlpP5kiqIRZy53GxcsAizyh6IGDdGNwgkwug3VkcIJMMIV3YWt1MgU
2026-03-12 04:35:20.806 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.175.121 waku 683c2ef6c39dcb4b61117daa276872d6b99b21535e919d58a536b0ad9feee0d0
2026-03-12 04:35:20.841 DEBUG [src.node.docker_mananger] Container started with ID 683c2ef6c39d. Setting up logs at ./log/docker/node2_2026-03-12_04-35-19__49444a02-ea45-40c8-8320-108aefb411cc__wakuorg_nwaku:latest.log
2026-03-12 04:35:20.842 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 5681
2026-03-12 04:35:20.842 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-12 04:35:21.842 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5681/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:21.845 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:21.946 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5681/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:21.948 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:22.049 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5681/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:22.052 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:22.152 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5681/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:22.155 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:22.255 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5681/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:22.258 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:22.358 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5681/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:22.361 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:22.461 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5681/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:22.466 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:22.566 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5681/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:22.569 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-12 04:35:22.669 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5681/health" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:22.676 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-12 04:35:22.676 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-12 04:35:22.676 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5681/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:22.683 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.175.121/tcp/5682/p2p/16Uiu2HAmE8U3HzQfpKkZya3UTkL2cgC5Him1A2j1xeJFRazsVqeS","/ip4/172.18.175.121/tcp/5683/ws/p2p/16Uiu2HAmE8U3HzQfpKkZya3UTkL2cgC5Him1A2j1xeJFRazsVqeS"],"enrUri":"enr:-L24QAG_rhlLhI07m0WgVcwN7i130muYnOTVbztuxXpN5R7rdepoPJzT6Dp9LUW3YcfVZhKCH9pKJ8GiwiTxHNGseZYCgmlkgnY0gmlwhKwSr3mKbXVsdGlhZGRyc5YACASsEq95BhYyAAoErBKveQYWM90DgnJzhQACAQAAiXNlY3AyNTZrMaEDFeWtecubNk6NlF_S5hWhngpbzPvH1fKRlgidBxvnt5uDdGNwghYyg3VkcIIWNIV3YWt1MgE"}'
2026-03-12 04:35:22.683 INFO [src.node.waku_node] REST service is ready !!
2026-03-12 04:35:22.683 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:5681/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.180.7/tcp/19502/p2p/16Uiu2HAm5qnuC1pLkb5T2bz9FhntgA7g539mkpP831nD8FopVtUg"]'
2026-03-12 04:35:22.704 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:35:22.708 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19501/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-12 04:35:22.728 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-12 04:35:22.729 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-12 04:35:22.830 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-12 04:35:22.830 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19501/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-12 04:35:22.833 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773290117777085378,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773290122708102260,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773290122766095417,"ephemeral":false,"proof":""}]'
2026-03-12 04:35:22.844 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-12 04:35:22.847 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-12 04:35:22.847 DEBUG [src.node.waku_node] Stopping container with id cbf49b1a2b51
2026-03-12 04:35:23.443 DEBUG [src.node.waku_node] Container stopped.
2026-03-12 04:35:23.445 DEBUG [src.node.waku_node] Stopping container with id 683c2ef6c39d
2026-03-12 04:35:24.055 DEBUG [src.node.waku_node] Container stopped.
2026-03-12 04:35:24.059 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-12 04:35:24.121 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-12 04:35:24.152 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-12 04:35:24.153 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-12 04:35:24.153 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-12 04:35:24.155 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-12_04-35-19__49444a02-ea45-40c8-8320-108aefb411cc__wakuorg_nwaku:latest.log
2026-03-12 04:35:24.157 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-12_04-35-19__49444a02-ea45-40c8-8320-108aefb411cc__wakuorg_nwaku:latest.log