277 lines
48 KiB
Plaintext

2026-03-14 04:34:17.371 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-14 04:34:17.371 DEBUG [tests.conftest] Running test: test_unsubscribe_from_all_pubsub_topics_one_by_one with id: 2026-03-14_04-34-17__20c028ff-45cf-4dd9-8e8a-e874155d7e6a
2026-03-14 04:34:17.371 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-14 04:34:17.372 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-14 04:34:17.372 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-14 04:34:17.379 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-14 04:34:17.379 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-14_04-34-17__20c028ff-45cf-4dd9-8e8a-e874155d7e6a__wakuorg_nwaku:latest.log
2026-03-14 04:34:17.379 DEBUG [src.node.waku_node] Starting Node...
2026-03-14 04:34:17.379 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-14 04:34:17.380 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-14 04:34:17.381 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.148.9
2026-03-14 04:34:17.381 DEBUG [src.node.docker_mananger] Generated ports ['54594', '54595', '54596', '54597', '54598']
2026-03-14 04:34:17.381 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-14 04:34:17.381 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-14 04:34:17.381 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-14 04:34:17.381 DEBUG [src.node.waku_node] Using volumes []
2026-03-14 04:34:17.381 DEBUG [src.node.docker_mananger] docker run -i -t -p 54594:54594 -p 54595:54595 -p 54596:54596 -p 54597:54597 -p 54598:54598 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=54596 --rest-port=54594 --tcp-port=54595 --discv5-udp-port=54597 --rest-address=0.0.0.0 --nat=extip:172.18.148.9 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=ad0a017a96e0a2f31fff8fd2dfdb9aa1f8badf01aad8d00a1bb31efee892fcd5 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=54598 --metrics-logging=true --relay=true --filter=true
2026-03-14 04:34:17.571 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.148.9 waku e981c3662392f0bcca36682afe4f85828e955b73d8f9d844005bc84be6eb03e7
2026-03-14 04:34:17.573 ERROR [src.node.docker_mananger] Max retries reached for container 149a9ec0c5f1. Exiting log stream.
2026-03-14 04:34:17.613 DEBUG [src.node.docker_mananger] Container started with ID e981c3662392. Setting up logs at ./log/docker/node1_2026-03-14_04-34-17__20c028ff-45cf-4dd9-8e8a-e874155d7e6a__wakuorg_nwaku:latest.log
2026-03-14 04:34:17.615 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 54594
2026-03-14 04:34:17.616 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-14 04:34:18.145 ERROR [src.node.docker_mananger] Max retries reached for container ee2d07a9df6d. Exiting log stream.
2026-03-14 04:34:18.616 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:54594/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:18.619 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-14 04:34:18.619 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-14 04:34:18.619 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:54594/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:18.621 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.148.9/tcp/54595/p2p/16Uiu2HAmDiWCvaeZiqw7hhBzUkaPfXs6UpM9yM4DNLjtufbWgQ5B","/ip4/172.18.148.9/tcp/54596/ws/p2p/16Uiu2HAmDiWCvaeZiqw7hhBzUkaPfXs6UpM9yM4DNLjtufbWgQ5B"],"enrUri":"enr:-L24QDtonhn_GNp1L-R56cBHnOIfpHNiJMWeXCPGGHULvCp9f2onWvoFXZluZ8daCMmQP2eou7leLf9WMgBb8kDONvoCgmlkgnY0gmlwhKwSlAmKbXVsdGlhZGRyc5YACASsEpQJBtVDAAoErBKUCQbVRN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDD8Iv-EEw2GwZ6ez0wgPAR-40NUtlcSmqmAwPmXsqf9aDdGNwgtVDg3VkcILVRYV3YWt1MgU"}'
2026-03-14 04:34:18.622 INFO [src.node.waku_node] REST service is ready !!
2026-03-14 04:34:18.628 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-14 04:34:18.628 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-14_04-34-17__20c028ff-45cf-4dd9-8e8a-e874155d7e6a__wakuorg_nwaku:latest.log
2026-03-14 04:34:18.629 DEBUG [src.node.waku_node] Starting Node...
2026-03-14 04:34:18.629 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-14 04:34:18.630 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-14 04:34:18.630 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.167.116
2026-03-14 04:34:18.630 DEBUG [src.node.docker_mananger] Generated ports ['26184', '26185', '26186', '26187', '26188']
2026-03-14 04:34:18.630 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-14 04:34:18.631 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-14 04:34:18.631 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-14 04:34:18.631 DEBUG [src.node.waku_node] Using volumes []
2026-03-14 04:34:18.631 DEBUG [src.node.docker_mananger] docker run -i -t -p 26184:26184 -p 26185:26185 -p 26186:26186 -p 26187:26187 -p 26188:26188 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=26186 --rest-port=26184 --tcp-port=26185 --discv5-udp-port=26187 --rest-address=0.0.0.0 --nat=extip:172.18.167.116 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=4e3efeab6e1b6c8c4714c6c13f6b3b2a07c223b642af3f54b30a87e880b01fcb --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=26188 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QDtonhn_GNp1L-R56cBHnOIfpHNiJMWeXCPGGHULvCp9f2onWvoFXZluZ8daCMmQP2eou7leLf9WMgBb8kDONvoCgmlkgnY0gmlwhKwSlAmKbXVsdGlhZGRyc5YACASsEpQJBtVDAAoErBKUCQbVRN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDD8Iv-EEw2GwZ6ez0wgPAR-40NUtlcSmqmAwPmXsqf9aDdGNwgtVDg3VkcILVRYV3YWt1MgU
2026-03-14 04:34:18.832 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.167.116 waku 628ac747014aed1a838740ca466ec9b96b1cddc50a50c1b6c6341f6d22fd8ac7
2026-03-14 04:34:18.867 DEBUG [src.node.docker_mananger] Container started with ID 628ac747014a. Setting up logs at ./log/docker/node2_2026-03-14_04-34-17__20c028ff-45cf-4dd9-8e8a-e874155d7e6a__wakuorg_nwaku:latest.log
2026-03-14 04:34:18.867 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 26184
2026-03-14 04:34:18.868 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-14 04:34:19.868 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:26184/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:19.871 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:19.971 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:26184/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:19.974 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:20.074 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:26184/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:20.076 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-14 04:34:20.077 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-14 04:34:20.077 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:26184/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:20.079 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.167.116/tcp/26185/p2p/16Uiu2HAm8Eo7traMnZFeUZiEjgHduYhA8XYNHTcAUV6L2dmE4fAP","/ip4/172.18.167.116/tcp/26186/ws/p2p/16Uiu2HAm8Eo7traMnZFeUZiEjgHduYhA8XYNHTcAUV6L2dmE4fAP"],"enrUri":"enr:-L24QMbC5vP1cjmK12go0Wsn4OG-IhMPe8xhdaV8NCq96x9bX6lbrwnBCnUm5mjUnVPXLh3XF_MrGW_EZZ5jDMDy-s8CgmlkgnY0gmlwhKwSp3SKbXVsdGlhZGRyc5YACASsEqd0BmZJAAoErBKndAZmSt0DgnJzhQACAQAAiXNlY3AyNTZrMaECvl6dL0prnYe5XJdLyObZKu9tJTgweeCOy9eFQxMwXcCDdGNwgmZJg3VkcIJmS4V3YWt1MgE"}'
2026-03-14 04:34:20.079 INFO [src.node.waku_node] REST service is ready !!
2026-03-14 04:34:20.080 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:26184/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.148.9/tcp/54595/p2p/16Uiu2HAmDiWCvaeZiqw7hhBzUkaPfXs6UpM9yM4DNLjtufbWgQ5B"]'
2026-03-14 04:34:20.108 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-14 04:34:20.109 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:54594/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-14 04:34:20.114 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-14 04:34:20.115 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:26184/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-14 04:34:20.121 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-14 04:34:20.122 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:54594/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-14 04:34:20.134 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-14 04:34:20.135 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-14 04:34:20.235 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-14 04:34:20.235 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:54594/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:20.238 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773462856294291586,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773462856005474628,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773462860122293049,"ephemeral":false,"proof":""}]'
2026-03-14 04:34:20.248 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-14 04:34:20.249 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-14 04:34:20.249 DEBUG [src.node.waku_node] Stopping container with id e981c3662392
2026-03-14 04:34:20.796 DEBUG [src.node.waku_node] Container stopped.
2026-03-14 04:34:20.797 DEBUG [src.node.waku_node] Stopping container with id 628ac747014a
2026-03-14 04:34:21.323 DEBUG [src.node.waku_node] Container stopped.
2026-03-14 04:34:21.326 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-14 04:34:21.357 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-14 04:34:21.384 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-14 04:34:21.386 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-14 04:34:21.386 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-14 04:34:21.388 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-14_04-34-17__20c028ff-45cf-4dd9-8e8a-e874155d7e6a__wakuorg_nwaku:latest.log
2026-03-14 04:34:21.391 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-14_04-34-17__20c028ff-45cf-4dd9-8e8a-e874155d7e6a__wakuorg_nwaku:latest.log
2026-03-14 04:34:21.407 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-14 04:34:21.409 DEBUG [tests.conftest] Running test: test_unsubscribe_from_all_pubsub_topics_one_by_one with id: 2026-03-14_04-34-21__60117731-c9d9-4de7-8a59-fb03ca66a6e4
2026-03-14 04:34:21.411 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-14 04:34:21.411 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-14 04:34:21.412 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-14 04:34:21.424 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-14 04:34:21.424 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-14_04-34-21__60117731-c9d9-4de7-8a59-fb03ca66a6e4__wakuorg_nwaku:latest.log
2026-03-14 04:34:21.424 DEBUG [src.node.waku_node] Starting Node...
2026-03-14 04:34:21.425 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-14 04:34:21.427 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-14 04:34:21.428 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.123.96
2026-03-14 04:34:21.428 DEBUG [src.node.docker_mananger] Generated ports ['13145', '13146', '13147', '13148', '13149']
2026-03-14 04:34:21.428 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-14 04:34:21.429 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-14 04:34:21.430 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-14 04:34:21.430 DEBUG [src.node.waku_node] Using volumes []
2026-03-14 04:34:21.431 DEBUG [src.node.docker_mananger] docker run -i -t -p 13145:13145 -p 13146:13146 -p 13147:13147 -p 13148:13148 -p 13149:13149 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=13147 --rest-port=13145 --tcp-port=13146 --discv5-udp-port=13148 --rest-address=0.0.0.0 --nat=extip:172.18.123.96 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=a859542d4fd3c3f978a24bdaaedcc55efbbdd9b6b66edf3a21dfe48c9b2de36f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=13149 --metrics-logging=true --relay=true --filter=true
2026-03-14 04:34:21.613 ERROR [src.node.docker_mananger] Max retries reached for container e981c3662392. Exiting log stream.
2026-03-14 04:34:21.638 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.123.96 waku f5c530a9983abd5637526880549e2cd758734b5946521b62ddec35822a7597b8
2026-03-14 04:34:21.673 DEBUG [src.node.docker_mananger] Container started with ID f5c530a9983a. Setting up logs at ./log/docker/node1_2026-03-14_04-34-21__60117731-c9d9-4de7-8a59-fb03ca66a6e4__wakuorg_nwaku:latest.log
2026-03-14 04:34:21.673 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 13145
2026-03-14 04:34:21.673 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-14 04:34:22.317 ERROR [src.node.docker_mananger] Max retries reached for container 628ac747014a. Exiting log stream.
2026-03-14 04:34:22.675 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13145/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:22.677 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:22.778 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13145/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:22.780 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:22.880 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13145/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:22.883 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:22.983 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13145/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:22.985 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:23.086 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13145/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:23.108 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:23.208 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13145/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:23.211 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-14 04:34:23.211 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-14 04:34:23.211 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13145/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:23.213 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.123.96/tcp/13146/p2p/16Uiu2HAmQg6wM8TGqpBMhg8bkuVmSRX6ocWWuFZKJSDNZnoazh88","/ip4/172.18.123.96/tcp/13147/ws/p2p/16Uiu2HAmQg6wM8TGqpBMhg8bkuVmSRX6ocWWuFZKJSDNZnoazh88"],"enrUri":"enr:-L24QAdNOQ8IP3z9g39-I8U03VYJWxhDSt_WtEoheuL8c_EubxPLEKb0suj9mx6PVwR9BUmHVsB-e1w7TSKYgq0m5yUCgmlkgnY0gmlwhKwSe2CKbXVsdGlhZGRyc5YACASsEntgBjNaAAoErBJ7YAYzW90DgnJzhQACAQAAiXNlY3AyNTZrMaEDspVFqJRFPi9xUZpztmHr10en56k_rVvqMAo54wWY9jWDdGNwgjNag3VkcIIzXIV3YWt1MgU"}'
2026-03-14 04:34:23.213 INFO [src.node.waku_node] REST service is ready !!
2026-03-14 04:34:23.220 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-14 04:34:23.221 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-14_04-34-21__60117731-c9d9-4de7-8a59-fb03ca66a6e4__wakuorg_nwaku:latest.log
2026-03-14 04:34:23.221 DEBUG [src.node.waku_node] Starting Node...
2026-03-14 04:34:23.221 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-14 04:34:23.222 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-14 04:34:23.222 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.80.114
2026-03-14 04:34:23.222 DEBUG [src.node.docker_mananger] Generated ports ['16584', '16585', '16586', '16587', '16588']
2026-03-14 04:34:23.223 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-14 04:34:23.223 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-14 04:34:23.223 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-14 04:34:23.223 DEBUG [src.node.waku_node] Using volumes []
2026-03-14 04:34:23.223 DEBUG [src.node.docker_mananger] docker run -i -t -p 16584:16584 -p 16585:16585 -p 16586:16586 -p 16587:16587 -p 16588:16588 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=16586 --rest-port=16584 --tcp-port=16585 --discv5-udp-port=16587 --rest-address=0.0.0.0 --nat=extip:172.18.80.114 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=ae7d6fbc0987f4156e9a5ca1f1c47e08df346abbbf58efa7b14abed5a064faea --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=16588 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QAdNOQ8IP3z9g39-I8U03VYJWxhDSt_WtEoheuL8c_EubxPLEKb0suj9mx6PVwR9BUmHVsB-e1w7TSKYgq0m5yUCgmlkgnY0gmlwhKwSe2CKbXVsdGlhZGRyc5YACASsEntgBjNaAAoErBJ7YAYzW90DgnJzhQACAQAAiXNlY3AyNTZrMaEDspVFqJRFPi9xUZpztmHr10en56k_rVvqMAo54wWY9jWDdGNwgjNag3VkcIIzXIV3YWt1MgU
2026-03-14 04:34:23.421 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.80.114 waku 92472e3411fa6ec6a4e9f9ed0ee412417bb9913cc4022f5f6d887907a7649596
2026-03-14 04:34:23.460 DEBUG [src.node.docker_mananger] Container started with ID 92472e3411fa. Setting up logs at ./log/docker/node2_2026-03-14_04-34-21__60117731-c9d9-4de7-8a59-fb03ca66a6e4__wakuorg_nwaku:latest.log
2026-03-14 04:34:23.460 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 16584
2026-03-14 04:34:23.461 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-14 04:34:24.461 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16584/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:24.464 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:24.564 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16584/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:24.567 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:24.668 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16584/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:24.700 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:24.801 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16584/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:24.803 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-14 04:34:24.803 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-14 04:34:24.803 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16584/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:24.806 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.80.114/tcp/16585/p2p/16Uiu2HAkvmufEbWGSDvbUKjB97eSbbvxPAgZ6kB4RCLMqkmALrNs","/ip4/172.18.80.114/tcp/16586/ws/p2p/16Uiu2HAkvmufEbWGSDvbUKjB97eSbbvxPAgZ6kB4RCLMqkmALrNs"],"enrUri":"enr:-L24QJMABbnx2Cq92_NWYBkDyDYMqV4-gQxWx547qnOtCaVBVR7Na6qC6veMtfCZwPyoW7kOHS3-3DA_92WxOe_oqMICgmlkgnY0gmlwhKwSUHKKbXVsdGlhZGRyc5YACASsElByBkDJAAoErBJQcgZAyt0DgnJzhQACAQAAiXNlY3AyNTZrMaECFArFayXZ_orKVvCIPqEN0GuiDb_mAd7rJEH0Cq9Y6aCDdGNwgkDJg3VkcIJAy4V3YWt1MgE"}'
2026-03-14 04:34:24.806 INFO [src.node.waku_node] REST service is ready !!
2026-03-14 04:34:24.807 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16584/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.123.96/tcp/13146/p2p/16Uiu2HAmQg6wM8TGqpBMhg8bkuVmSRX6ocWWuFZKJSDNZnoazh88"]'
2026-03-14 04:34:24.837 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-14 04:34:24.838 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13145/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-14 04:34:24.846 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-14 04:34:24.847 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16584/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-14 04:34:24.855 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-14 04:34:24.856 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13145/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-14 04:34:24.869 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-14 04:34:24.869 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-14 04:34:24.970 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-14 04:34:24.970 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13145/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:24.973 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773462860436085091,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773462860280524209,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773462864654333411,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773462864712451289,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773462864856799938,"ephemeral":false,"proof":""}]'
2026-03-14 04:34:24.983 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-14 04:34:24.985 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-14 04:34:24.985 DEBUG [src.node.waku_node] Stopping container with id f5c530a9983a
2026-03-14 04:34:25.568 DEBUG [src.node.waku_node] Container stopped.
2026-03-14 04:34:25.569 DEBUG [src.node.waku_node] Stopping container with id 92472e3411fa
2026-03-14 04:34:26.142 DEBUG [src.node.waku_node] Container stopped.
2026-03-14 04:34:26.145 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-14 04:34:26.177 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-14 04:34:26.209 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-14 04:34:26.210 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-14 04:34:26.210 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-14 04:34:26.214 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-14_04-34-21__60117731-c9d9-4de7-8a59-fb03ca66a6e4__wakuorg_nwaku:latest.log
2026-03-14 04:34:26.217 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-14_04-34-21__60117731-c9d9-4de7-8a59-fb03ca66a6e4__wakuorg_nwaku:latest.log
2026-03-14 04:34:26.232 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-14 04:34:26.234 DEBUG [tests.conftest] Running test: test_unsubscribe_from_all_pubsub_topics_one_by_one with id: 2026-03-14_04-34-26__bdb1ecea-71f4-4234-bf1c-5a4629376bf3
2026-03-14 04:34:26.235 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-14 04:34:26.236 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-14 04:34:26.236 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-14 04:34:26.245 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-14 04:34:26.246 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-14_04-34-26__bdb1ecea-71f4-4234-bf1c-5a4629376bf3__wakuorg_nwaku:latest.log
2026-03-14 04:34:26.246 DEBUG [src.node.waku_node] Starting Node...
2026-03-14 04:34:26.246 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-14 04:34:26.249 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-14 04:34:26.250 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.106.107
2026-03-14 04:34:26.251 DEBUG [src.node.docker_mananger] Generated ports ['49224', '49225', '49226', '49227', '49228']
2026-03-14 04:34:26.251 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-14 04:34:26.251 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-14 04:34:26.252 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-14 04:34:26.252 DEBUG [src.node.waku_node] Using volumes []
2026-03-14 04:34:26.253 DEBUG [src.node.docker_mananger] docker run -i -t -p 49224:49224 -p 49225:49225 -p 49226:49226 -p 49227:49227 -p 49228:49228 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=49226 --rest-port=49224 --tcp-port=49225 --discv5-udp-port=49227 --rest-address=0.0.0.0 --nat=extip:172.18.106.107 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=3e6fc2142b68ecaabc740fb9e56d50ff0663393cd69aa3f746725a7d9dc3ff3b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=49228 --metrics-logging=true --relay=true --filter=true
2026-03-14 04:34:26.415 ERROR [src.node.docker_mananger] Max retries reached for container f5c530a9983a. Exiting log stream.
2026-03-14 04:34:26.463 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.106.107 waku 61e342be282eb878877a223d9401e8a49123303c2c270b71b609fa59b9f1fc70
2026-03-14 04:34:26.500 DEBUG [src.node.docker_mananger] Container started with ID 61e342be282e. Setting up logs at ./log/docker/node1_2026-03-14_04-34-26__bdb1ecea-71f4-4234-bf1c-5a4629376bf3__wakuorg_nwaku:latest.log
2026-03-14 04:34:26.501 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 49224
2026-03-14 04:34:26.501 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-14 04:34:27.159 ERROR [src.node.docker_mananger] Max retries reached for container 92472e3411fa. Exiting log stream.
2026-03-14 04:34:27.501 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:49224/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:27.504 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:27.604 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:49224/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:27.607 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:27.707 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:49224/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:27.710 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:27.811 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:49224/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:27.813 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:27.913 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:49224/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:27.916 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:28.016 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:49224/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:28.018 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:28.118 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:49224/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:28.121 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:28.221 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:49224/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:28.224 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:28.324 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:49224/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:28.327 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-14 04:34:28.327 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-14 04:34:28.327 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:49224/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:28.330 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.106.107/tcp/49225/p2p/16Uiu2HAkwiJkMkdrUmxkJJcJb8niXxEWFJKFARAyq6rmyUcrbXt9","/ip4/172.18.106.107/tcp/49226/ws/p2p/16Uiu2HAkwiJkMkdrUmxkJJcJb8niXxEWFJKFARAyq6rmyUcrbXt9"],"enrUri":"enr:-L24QKwH2n6MBckhfR4CESSyeOZzRklZwDzjlFBFDL9PuXt8PJSAypIf_8SvM8MK4hN_c9csQzicu3x5ARe6xHNxyokCgmlkgnY0gmlwhKwSamuKbXVsdGlhZGRyc5YACASsEmprBsBJAAoErBJqawbASt0DgnJzhQACAQAAiXNlY3AyNTZrMaECIfo9rOQD0uXijx6h-ByJ-1yRh4InR67hWx0rvvuQ8s6DdGNwgsBJg3VkcILAS4V3YWt1MgU"}'
2026-03-14 04:34:28.330 INFO [src.node.waku_node] REST service is ready !!
2026-03-14 04:34:28.336 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-14 04:34:28.337 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-14_04-34-26__bdb1ecea-71f4-4234-bf1c-5a4629376bf3__wakuorg_nwaku:latest.log
2026-03-14 04:34:28.337 DEBUG [src.node.waku_node] Starting Node...
2026-03-14 04:34:28.337 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-14 04:34:28.338 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-14 04:34:28.338 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.169.186
2026-03-14 04:34:28.338 DEBUG [src.node.docker_mananger] Generated ports ['16750', '16751', '16752', '16753', '16754']
2026-03-14 04:34:28.339 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-14 04:34:28.339 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-14 04:34:28.339 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-14 04:34:28.339 DEBUG [src.node.waku_node] Using volumes []
2026-03-14 04:34:28.339 DEBUG [src.node.docker_mananger] docker run -i -t -p 16750:16750 -p 16751:16751 -p 16752:16752 -p 16753:16753 -p 16754:16754 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=16752 --rest-port=16750 --tcp-port=16751 --discv5-udp-port=16753 --rest-address=0.0.0.0 --nat=extip:172.18.169.186 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=a1da13feb0eadd2eb9c87b2c4a5f261c89ac3e059cddebfc33a2977b5db9e153 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=16754 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QKwH2n6MBckhfR4CESSyeOZzRklZwDzjlFBFDL9PuXt8PJSAypIf_8SvM8MK4hN_c9csQzicu3x5ARe6xHNxyokCgmlkgnY0gmlwhKwSamuKbXVsdGlhZGRyc5YACASsEmprBsBJAAoErBJqawbASt0DgnJzhQACAQAAiXNlY3AyNTZrMaECIfo9rOQD0uXijx6h-ByJ-1yRh4InR67hWx0rvvuQ8s6DdGNwgsBJg3VkcILAS4V3YWt1MgU
2026-03-14 04:34:28.548 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.169.186 waku ff87ef2cb79ec4e131a851be11a1c4db409c6c51ee21a471128744d59983839c
2026-03-14 04:34:28.585 DEBUG [src.node.docker_mananger] Container started with ID ff87ef2cb79e. Setting up logs at ./log/docker/node2_2026-03-14_04-34-26__bdb1ecea-71f4-4234-bf1c-5a4629376bf3__wakuorg_nwaku:latest.log
2026-03-14 04:34:28.586 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 16750
2026-03-14 04:34:28.586 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-14 04:34:29.587 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16750/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:29.590 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:29.690 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16750/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:29.693 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:29.793 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16750/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:29.795 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:29.896 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16750/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:29.898 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:29.998 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16750/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:30.000 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:30.101 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16750/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:30.103 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-14 04:34:30.204 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16750/health" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:30.206 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-14 04:34:30.206 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-14 04:34:30.206 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16750/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:30.208 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.169.186/tcp/16751/p2p/16Uiu2HAmVqCYw4uZ2aGTZXsBBuiEaq4f97J2Bq4Fkx9K92YSuggr","/ip4/172.18.169.186/tcp/16752/ws/p2p/16Uiu2HAmVqCYw4uZ2aGTZXsBBuiEaq4f97J2Bq4Fkx9K92YSuggr"],"enrUri":"enr:-L24QBa9sRxlgZwyeImXy65qHyt1dycafd__Tcz6mEFEa4aXD1Od_NiaMtd4hx24Utrm5Dqq-iKYSQgwHkSGh9CjAegCgmlkgnY0gmlwhKwSqbqKbXVsdGlhZGRyc5YACASsEqm6BkFvAAoErBKpugZBcN0DgnJzhQACAQAAiXNlY3AyNTZrMaED_zRIAOUBm0dvp7SwzLWvHYka9foGte7gmFIrr00TjtODdGNwgkFvg3VkcIJBcYV3YWt1MgE"}'
2026-03-14 04:34:30.209 INFO [src.node.waku_node] REST service is ready !!
2026-03-14 04:34:30.209 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16750/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.106.107/tcp/49225/p2p/16Uiu2HAkwiJkMkdrUmxkJJcJb8niXxEWFJKFARAyq6rmyUcrbXt9"]'
2026-03-14 04:34:30.245 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-14 04:34:30.246 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:49224/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-14 04:34:30.252 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-14 04:34:30.253 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16750/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-14 04:34:30.259 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-14 04:34:30.260 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:49224/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-14 04:34:30.270 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-14 04:34:30.270 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-14 04:34:30.371 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-14 04:34:30.371 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:49224/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-14 04:34:30.374 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773462870143313802,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773462870260639039,"ephemeral":false,"proof":""}]'
2026-03-14 04:34:30.383 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-14 04:34:30.385 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-14 04:34:30.385 DEBUG [src.node.waku_node] Stopping container with id 61e342be282e
2026-03-14 04:34:30.971 DEBUG [src.node.waku_node] Container stopped.
2026-03-14 04:34:30.972 DEBUG [src.node.waku_node] Stopping container with id ff87ef2cb79e
2026-03-14 04:34:31.531 DEBUG [src.node.waku_node] Container stopped.
2026-03-14 04:34:31.537 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-14 04:34:31.575 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-14 04:34:31.605 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-14 04:34:31.606 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-14 04:34:31.607 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-14 04:34:31.608 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-14_04-34-26__bdb1ecea-71f4-4234-bf1c-5a4629376bf3__wakuorg_nwaku:latest.log
2026-03-14 04:34:31.609 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-14_04-34-26__bdb1ecea-71f4-4234-bf1c-5a4629376bf3__wakuorg_nwaku:latest.log