261 lines
46 KiB
Plaintext

2026-03-18 04:43:18.128 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-18 04:43:18.128 DEBUG [tests.conftest] Running test: test_unsubscribe_from_all_content_topics with id: 2026-03-18_04-43-18__f05f113e-f2b5-4aae-884e-5353609680c0
2026-03-18 04:43:18.128 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-18 04:43:18.129 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-18 04:43:18.129 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-18 04:43:18.137 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:43:18.137 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-18_04-43-18__f05f113e-f2b5-4aae-884e-5353609680c0__wakuorg_nwaku:latest.log
2026-03-18 04:43:18.137 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:43:18.137 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:43:18.139 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:43:18.139 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.254.112
2026-03-18 04:43:18.139 DEBUG [src.node.docker_mananger] Generated ports ['12838', '12839', '12840', '12841', '12842']
2026-03-18 04:43:18.139 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:43:18.139 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:43:18.139 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:43:18.140 DEBUG [src.node.docker_mananger] docker run -i -t -p 12838:12838 -p 12839:12839 -p 12840:12840 -p 12841:12841 -p 12842:12842 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=12840 --rest-port=12838 --tcp-port=12839 --discv5-udp-port=12841 --rest-address=0.0.0.0 --nat=extip:172.18.254.112 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=ebdbac9a1ee71cdde2e48c9044fb33364fbd6c7a1c9ceb07a92d8a5ad7d88d67 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=12842 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
2026-03-18 04:43:18.329 ERROR [src.node.docker_mananger] Max retries reached for container a5bf42cf097f. Exiting log stream.
2026-03-18 04:43:18.331 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.254.112 waku 7a59dbf92b76e9c031f7f73f1242327b0e9510568ba3153d52378e07f1efed59
2026-03-18 04:43:18.365 DEBUG [src.node.docker_mananger] Container started with ID 7a59dbf92b76. Setting up logs at ./log/docker/node1_2026-03-18_04-43-18__f05f113e-f2b5-4aae-884e-5353609680c0__wakuorg_nwaku:latest.log
2026-03-18 04:43:18.365 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 12838
2026-03-18 04:43:18.366 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:43:18.877 ERROR [src.node.docker_mananger] Max retries reached for container ae2c4811285a. Exiting log stream.
2026-03-18 04:43:19.366 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:12838/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:19.369 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:19.469 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:12838/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:19.472 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:19.572 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:12838/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:19.575 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:43:19.575 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:43:19.576 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:12838/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:19.578 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.254.112/tcp/12839/p2p/16Uiu2HAmAf37SzjwGtNCBx6s7rSQqZMY7iWYAKsTXJQ8R3BMX8ip","/ip4/172.18.254.112/tcp/12840/ws/p2p/16Uiu2HAmAf37SzjwGtNCBx6s7rSQqZMY7iWYAKsTXJQ8R3BMX8ip"],"enrUri":"enr:-L24QIHOQQ17x1HVArfKYQ7V6bgjl80_KAjwLdOYxjE4CVy_GHI0Ku9U3OT7if8PqJ1mvquWzTEv8zuy-DdnErT8arkCgmlkgnY0gmlwhKwS_nCKbXVsdGlhZGRyc5YACASsEv5wBjInAAoErBL-cAYyKN0DgnJzhQACAQAAiXNlY3AyNTZrMaEC4ku_mT4s2jWwPT37nsNnmq0c8f2YJecLjeyk4NjGbSWDdGNwgjIng3VkcIIyKYV3YWt1MgU"}'
2026-03-18 04:43:19.578 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:43:19.586 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:43:19.586 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-18_04-43-18__f05f113e-f2b5-4aae-884e-5353609680c0__wakuorg_nwaku:latest.log
2026-03-18 04:43:19.587 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:43:19.587 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:43:19.590 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:43:19.590 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.101.60
2026-03-18 04:43:19.590 DEBUG [src.node.docker_mananger] Generated ports ['28989', '28990', '28991', '28992', '28993']
2026-03-18 04:43:19.590 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:43:19.590 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:43:19.590 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:43:19.591 DEBUG [src.node.docker_mananger] docker run -i -t -p 28989:28989 -p 28990:28990 -p 28991:28991 -p 28992:28992 -p 28993:28993 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=28991 --rest-port=28989 --tcp-port=28990 --discv5-udp-port=28992 --rest-address=0.0.0.0 --nat=extip:172.18.101.60 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=ae60cdb7e8eca4ca9fab18824970d0d1a29fec78c0ef9f20e316f9c5af352e26 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=28993 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QIHOQQ17x1HVArfKYQ7V6bgjl80_KAjwLdOYxjE4CVy_GHI0Ku9U3OT7if8PqJ1mvquWzTEv8zuy-DdnErT8arkCgmlkgnY0gmlwhKwS_nCKbXVsdGlhZGRyc5YACASsEv5wBjInAAoErBL-cAYyKN0DgnJzhQACAQAAiXNlY3AyNTZrMaEC4ku_mT4s2jWwPT37nsNnmq0c8f2YJecLjeyk4NjGbSWDdGNwgjIng3VkcIIyKYV3YWt1MgU --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
2026-03-18 04:43:19.807 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.101.60 waku eae22bb3710c898a046a6b5ae4292144161cc314a6df3da4fbdf166d2c08ae2c
2026-03-18 04:43:19.846 DEBUG [src.node.docker_mananger] Container started with ID eae22bb3710c. Setting up logs at ./log/docker/node2_2026-03-18_04-43-18__f05f113e-f2b5-4aae-884e-5353609680c0__wakuorg_nwaku:latest.log
2026-03-18 04:43:19.846 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 28989
2026-03-18 04:43:19.847 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:43:20.847 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:28989/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:20.850 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:20.950 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:28989/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:20.954 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:43:20.954 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:43:20.954 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:28989/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:20.956 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.101.60/tcp/28990/p2p/16Uiu2HAmDBuis5Xtod6addRwCZd3QjzCrDjdZjYxLGfpGxiYhMWA","/ip4/172.18.101.60/tcp/28991/ws/p2p/16Uiu2HAmDBuis5Xtod6addRwCZd3QjzCrDjdZjYxLGfpGxiYhMWA"],"enrUri":"enr:-L24QHRc9Qn8G_oSRvpwR1fmjDgEHd0p4IhXe3lNnqbWwu1qen4wPBP-35TuG4KATH9KecfvY9JyQmEkEhlwIZmhs_gCgmlkgnY0gmlwhKwSZTyKbXVsdGlhZGRyc5YACASsEmU8BnE-AAoErBJlPAZxP90DgnJzhQACAQAAiXNlY3AyNTZrMaEDB-vFq9HMuWOho6IEAkzBiYuFzsjd8cmazDqp26tmc3uDdGNwgnE-g3VkcIJxQIV3YWt1MgE"}'
2026-03-18 04:43:20.957 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:43:20.957 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:28989/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.254.112/tcp/12839/p2p/16Uiu2HAmAf37SzjwGtNCBx6s7rSQqZMY7iWYAKsTXJQ8R3BMX8ip"]'
2026-03-18 04:43:20.980 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:43:20.982 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12838/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
2026-03-18 04:43:20.998 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:43:20.999 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:28989/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
2026-03-18 04:43:21.009 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:43:21.010 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12838/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:43:21.022 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:43:21.022 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-18 04:43:21.124 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-18 04:43:21.124 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:12838/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:21.127 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773809001010231966,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773809000967428144,"ephemeral":false,"proof":""}]'
2026-03-18 04:43:21.174 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-18 04:43:21.175 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-18 04:43:21.175 DEBUG [src.node.waku_node] Stopping container with id 7a59dbf92b76
2026-03-18 04:43:21.759 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:43:21.760 DEBUG [src.node.waku_node] Stopping container with id eae22bb3710c
2026-03-18 04:43:22.317 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:43:22.319 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-18 04:43:22.351 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:43:22.378 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:43:22.380 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-18 04:43:22.380 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-18 04:43:22.382 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-18_04-43-18__f05f113e-f2b5-4aae-884e-5353609680c0__wakuorg_nwaku:latest.log
2026-03-18 04:43:22.383 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-18_04-43-18__f05f113e-f2b5-4aae-884e-5353609680c0__wakuorg_nwaku:latest.log
2026-03-18 04:43:22.387 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-18 04:43:22.387 DEBUG [tests.conftest] Running test: test_unsubscribe_from_all_content_topics with id: 2026-03-18_04-43-22__7e876e8e-25fb-4934-9a52-3e8d3bdbd8ae
2026-03-18 04:43:22.388 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-18 04:43:22.388 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-18 04:43:22.388 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-18 04:43:22.395 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:43:22.396 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-18_04-43-22__7e876e8e-25fb-4934-9a52-3e8d3bdbd8ae__wakuorg_nwaku:latest.log
2026-03-18 04:43:22.396 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:43:22.396 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:43:22.397 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:43:22.397 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.67.75
2026-03-18 04:43:22.397 DEBUG [src.node.docker_mananger] Generated ports ['29661', '29662', '29663', '29664', '29665']
2026-03-18 04:43:22.397 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:43:22.397 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:43:22.398 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:43:22.398 DEBUG [src.node.docker_mananger] docker run -i -t -p 29661:29661 -p 29662:29662 -p 29663:29663 -p 29664:29664 -p 29665:29665 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=29663 --rest-port=29661 --tcp-port=29662 --discv5-udp-port=29664 --rest-address=0.0.0.0 --nat=extip:172.18.67.75 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=0dbefe99ba39bf36e4e4ecfaecca546a3a117f0d50e0cede7c6e2ef0ab1cb9a7 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=29665 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
2026-03-18 04:43:22.585 ERROR [src.node.docker_mananger] Max retries reached for container 7a59dbf92b76. Exiting log stream.
2026-03-18 04:43:22.586 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.67.75 waku b284894f7d947f601f2e39c0fd290887f99344aad9c9dd393cc8536fb0606abc
2026-03-18 04:43:22.623 DEBUG [src.node.docker_mananger] Container started with ID b284894f7d94. Setting up logs at ./log/docker/node1_2026-03-18_04-43-22__7e876e8e-25fb-4934-9a52-3e8d3bdbd8ae__wakuorg_nwaku:latest.log
2026-03-18 04:43:22.623 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 29661
2026-03-18 04:43:22.623 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:43:23.190 ERROR [src.node.docker_mananger] Max retries reached for container eae22bb3710c. Exiting log stream.
2026-03-18 04:43:23.625 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29661/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:23.628 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:23.728 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29661/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:23.731 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:23.831 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29661/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:23.834 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:23.935 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29661/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:23.942 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:24.042 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29661/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:24.045 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:43:24.045 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:43:24.045 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29661/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:24.048 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.67.75/tcp/29662/p2p/16Uiu2HAmBumZHz2RpEKqtXDehmzzdoTPzkJJrMePVT83e9svR2Sw","/ip4/172.18.67.75/tcp/29663/ws/p2p/16Uiu2HAmBumZHz2RpEKqtXDehmzzdoTPzkJJrMePVT83e9svR2Sw"],"enrUri":"enr:-L24QGiLgyxBzIyo6-lB6cNS2GUU9CulHIOLGjxY9IrYuICzUu1jx4awDdYevh06fFrJ4Aytpl_XNqi3_rxNZVyPteICgmlkgnY0gmlwhKwSQ0uKbXVsdGlhZGRyc5YACASsEkNLBnPeAAoErBJDSwZz390DgnJzhQACAQAAiXNlY3AyNTZrMaEC9O2PYQ9iYeEVUfDKZNsAz8euyu3pP9Tcy03BbdPrStSDdGNwgnPeg3VkcIJz4IV3YWt1MgU"}'
2026-03-18 04:43:24.048 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:43:24.055 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:43:24.056 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-18_04-43-22__7e876e8e-25fb-4934-9a52-3e8d3bdbd8ae__wakuorg_nwaku:latest.log
2026-03-18 04:43:24.056 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:43:24.056 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:43:24.057 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:43:24.057 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.232.113
2026-03-18 04:43:24.057 DEBUG [src.node.docker_mananger] Generated ports ['45502', '45503', '45504', '45505', '45506']
2026-03-18 04:43:24.058 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:43:24.058 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:43:24.058 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:43:24.058 DEBUG [src.node.docker_mananger] docker run -i -t -p 45502:45502 -p 45503:45503 -p 45504:45504 -p 45505:45505 -p 45506:45506 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=45504 --rest-port=45502 --tcp-port=45503 --discv5-udp-port=45505 --rest-address=0.0.0.0 --nat=extip:172.18.232.113 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=bdfde1eefada731bcc664f7f0fdac4bebf9ed84f6bfeb8dcd9a7dcf1ca535bda --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=45506 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QGiLgyxBzIyo6-lB6cNS2GUU9CulHIOLGjxY9IrYuICzUu1jx4awDdYevh06fFrJ4Aytpl_XNqi3_rxNZVyPteICgmlkgnY0gmlwhKwSQ0uKbXVsdGlhZGRyc5YACASsEkNLBnPeAAoErBJDSwZz390DgnJzhQACAQAAiXNlY3AyNTZrMaEC9O2PYQ9iYeEVUfDKZNsAz8euyu3pP9Tcy03BbdPrStSDdGNwgnPeg3VkcIJz4IV3YWt1MgU --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
2026-03-18 04:43:24.269 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.232.113 waku 63a905ba2821de3a9060fe8eced449394cf6a4c0c1c4d241c2ae5ef326305d5b
2026-03-18 04:43:24.308 DEBUG [src.node.docker_mananger] Container started with ID 63a905ba2821. Setting up logs at ./log/docker/node2_2026-03-18_04-43-22__7e876e8e-25fb-4934-9a52-3e8d3bdbd8ae__wakuorg_nwaku:latest.log
2026-03-18 04:43:24.309 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 45502
2026-03-18 04:43:24.309 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:43:25.309 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:45502/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:25.312 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:25.412 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:45502/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:25.415 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:25.515 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:45502/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:25.518 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:25.619 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:45502/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:25.621 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:43:25.622 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:43:25.622 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:45502/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:25.624 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.232.113/tcp/45503/p2p/16Uiu2HAmNqbwyxqx6gBZtEwKEJdhSwne6PJXq7HE11cuebBg4A8x","/ip4/172.18.232.113/tcp/45504/ws/p2p/16Uiu2HAmNqbwyxqx6gBZtEwKEJdhSwne6PJXq7HE11cuebBg4A8x"],"enrUri":"enr:-L24QNvMzz6_Scu5bJNm9y6G1pbnKuUve-2x2BIafzOk-crNVBbzn3n2U8QNebAXm-N_Vn4_h6EHWIP1KbkOHX_9hQwCgmlkgnY0gmlwhKwS6HGKbXVsdGlhZGRyc5YACASsEuhxBrG_AAoErBLocQaxwN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDl0ztpjvw8odGAGDuCfVePMWIIEhdH8nH5HyR5styRzmDdGNwgrG_g3VkcIKxwYV3YWt1MgE"}'
2026-03-18 04:43:25.625 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:43:25.625 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:45502/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.67.75/tcp/29662/p2p/16Uiu2HAmBumZHz2RpEKqtXDehmzzdoTPzkJJrMePVT83e9svR2Sw"]'
2026-03-18 04:43:25.644 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:43:25.648 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29661/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
2026-03-18 04:43:25.654 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:43:25.655 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:45502/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
2026-03-18 04:43:25.661 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:43:25.662 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29661/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:43:25.672 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:43:25.673 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-18 04:43:25.773 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-18 04:43:25.774 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29661/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:25.777 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773809005070677122,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773809005662465123,"ephemeral":false,"proof":""}]'
2026-03-18 04:43:25.788 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-18 04:43:25.789 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-18 04:43:25.790 DEBUG [src.node.waku_node] Stopping container with id b284894f7d94
2026-03-18 04:43:26.372 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:43:26.374 DEBUG [src.node.waku_node] Stopping container with id 63a905ba2821
2026-03-18 04:43:26.975 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:43:26.976 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-18 04:43:27.047 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:43:27.075 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:43:27.076 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-18 04:43:27.076 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-18 04:43:27.076 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-18_04-43-22__7e876e8e-25fb-4934-9a52-3e8d3bdbd8ae__wakuorg_nwaku:latest.log
2026-03-18 04:43:27.077 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-18_04-43-22__7e876e8e-25fb-4934-9a52-3e8d3bdbd8ae__wakuorg_nwaku:latest.log
2026-03-18 04:43:27.082 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-18 04:43:27.083 DEBUG [tests.conftest] Running test: test_unsubscribe_from_all_content_topics with id: 2026-03-18_04-43-27__22b1607d-5728-402e-aca9-c870d9ef75df
2026-03-18 04:43:27.083 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-18 04:43:27.083 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-18 04:43:27.083 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-18 04:43:27.091 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:43:27.091 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-18_04-43-27__22b1607d-5728-402e-aca9-c870d9ef75df__wakuorg_nwaku:latest.log
2026-03-18 04:43:27.091 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:43:27.092 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:43:27.093 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:43:27.093 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.37.41
2026-03-18 04:43:27.093 DEBUG [src.node.docker_mananger] Generated ports ['57309', '57310', '57311', '57312', '57313']
2026-03-18 04:43:27.094 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:43:27.094 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:43:27.094 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:43:27.094 DEBUG [src.node.docker_mananger] docker run -i -t -p 57309:57309 -p 57310:57310 -p 57311:57311 -p 57312:57312 -p 57313:57313 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=57311 --rest-port=57309 --tcp-port=57310 --discv5-udp-port=57312 --rest-address=0.0.0.0 --nat=extip:172.18.37.41 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=ac8af585f43e1cbafcf677efbf9c7d97a29dded82f7ccddebaa7d6f8f22cee7b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=57313 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
2026-03-18 04:43:27.293 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.37.41 waku f2890a929d0a635fcfb5586ff564663d159b4dabaea843bd94cfd7263c690fa1
2026-03-18 04:43:27.331 DEBUG [src.node.docker_mananger] Container started with ID f2890a929d0a. Setting up logs at ./log/docker/node1_2026-03-18_04-43-27__22b1607d-5728-402e-aca9-c870d9ef75df__wakuorg_nwaku:latest.log
2026-03-18 04:43:27.332 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 57309
2026-03-18 04:43:27.332 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:43:27.338 ERROR [src.node.docker_mananger] Max retries reached for container b284894f7d94. Exiting log stream.
2026-03-18 04:43:27.858 ERROR [src.node.docker_mananger] Max retries reached for container 63a905ba2821. Exiting log stream.
2026-03-18 04:43:28.332 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57309/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:28.335 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:28.436 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57309/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:28.439 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:28.539 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57309/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:28.542 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:28.642 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57309/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:28.645 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:28.746 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57309/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:28.749 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:28.849 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57309/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:28.852 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:28.952 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57309/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:28.955 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:29.056 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57309/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:29.058 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:43:29.058 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:43:29.059 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57309/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:29.061 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.37.41/tcp/57310/p2p/16Uiu2HAkvRgcTa4NtjJqbrcZNqzvNWpsauJdKq6oywLeNvRZ1nbU","/ip4/172.18.37.41/tcp/57311/ws/p2p/16Uiu2HAkvRgcTa4NtjJqbrcZNqzvNWpsauJdKq6oywLeNvRZ1nbU"],"enrUri":"enr:-L24QHkwk4644X0bRON4Ef4q72tEbwhm_wFdQetv2FNE7gbrTdwdjC1Oukjk2-XTKp3YOXJN9tAeX7EDnioKixzeWBsCgmlkgnY0gmlwhKwSJSmKbXVsdGlhZGRyc5YACASsEiUpBt_eAAoErBIlKQbf390DgnJzhQACAQAAiXNlY3AyNTZrMaECDtxm1sAJ_SQi-w4ObuUjlK8VgJ3Kfej8IINhjvoQkuODdGNwgt_eg3VkcILf4IV3YWt1MgU"}'
2026-03-18 04:43:29.061 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:43:29.069 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-18 04:43:29.069 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-18_04-43-27__22b1607d-5728-402e-aca9-c870d9ef75df__wakuorg_nwaku:latest.log
2026-03-18 04:43:29.069 DEBUG [src.node.waku_node] Starting Node...
2026-03-18 04:43:29.069 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-18 04:43:29.071 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-18 04:43:29.071 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.218.90
2026-03-18 04:43:29.071 DEBUG [src.node.docker_mananger] Generated ports ['33309', '33310', '33311', '33312', '33313']
2026-03-18 04:43:29.071 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-18 04:43:29.071 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-18 04:43:29.071 DEBUG [src.node.waku_node] Using volumes []
2026-03-18 04:43:29.072 DEBUG [src.node.docker_mananger] docker run -i -t -p 33309:33309 -p 33310:33310 -p 33311:33311 -p 33312:33312 -p 33313:33313 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=33311 --rest-port=33309 --tcp-port=33310 --discv5-udp-port=33312 --rest-address=0.0.0.0 --nat=extip:172.18.218.90 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=2901b6ff8fae21bd42bccb9c258c6d7d42bf962cffb3ec0826bd9f7cdbf8de97 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=33313 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QHkwk4644X0bRON4Ef4q72tEbwhm_wFdQetv2FNE7gbrTdwdjC1Oukjk2-XTKp3YOXJN9tAeX7EDnioKixzeWBsCgmlkgnY0gmlwhKwSJSmKbXVsdGlhZGRyc5YACASsEiUpBt_eAAoErBIlKQbf390DgnJzhQACAQAAiXNlY3AyNTZrMaECDtxm1sAJ_SQi-w4ObuUjlK8VgJ3Kfej8IINhjvoQkuODdGNwgt_eg3VkcILf4IV3YWt1MgU --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
2026-03-18 04:43:29.280 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.218.90 waku f16772bd829ec032cfdb0175f14e819bca71807c1a212a545adb0252905fc4be
2026-03-18 04:43:29.318 DEBUG [src.node.docker_mananger] Container started with ID f16772bd829e. Setting up logs at ./log/docker/node2_2026-03-18_04-43-27__22b1607d-5728-402e-aca9-c870d9ef75df__wakuorg_nwaku:latest.log
2026-03-18 04:43:29.318 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 33309
2026-03-18 04:43:29.318 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-18 04:43:30.319 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33309/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:30.321 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:30.422 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33309/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:30.425 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-18 04:43:30.525 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33309/health" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:30.530 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-18 04:43:30.530 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-18 04:43:30.530 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33309/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:30.538 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.218.90/tcp/33310/p2p/16Uiu2HAm24wNiBikS24AUtuvg5aphM5dfEFz3ac8uNz2AEoFrM6w","/ip4/172.18.218.90/tcp/33311/ws/p2p/16Uiu2HAm24wNiBikS24AUtuvg5aphM5dfEFz3ac8uNz2AEoFrM6w"],"enrUri":"enr:-L24QJBaXIMoCReUZ0GUvIwD_l5YPgV16dJSz4bbnJ15HCrRfqUCS-Njci8pzI7AzKIDNiCC0degefz3rzBfj7TLYNICgmlkgnY0gmlwhKwS2lqKbXVsdGlhZGRyc5YACASsEtpaBoIeAAoErBLaWgaCH90DgnJzhQACAQAAiXNlY3AyNTZrMaECYrIE0V_618rSIXLH9JrQAAn-mejAEOR9HUJ3fwU24jCDdGNwgoIeg3VkcIKCIIV3YWt1MgE"}'
2026-03-18 04:43:30.538 INFO [src.node.waku_node] REST service is ready !!
2026-03-18 04:43:30.539 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:33309/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.37.41/tcp/57310/p2p/16Uiu2HAkvRgcTa4NtjJqbrcZNqzvNWpsauJdKq6oywLeNvRZ1nbU"]'
2026-03-18 04:43:30.573 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:43:30.574 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57309/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
2026-03-18 04:43:30.587 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:43:30.589 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:33309/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
2026-03-18 04:43:30.597 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:43:30.598 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57309/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-18 04:43:30.610 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-18 04:43:30.611 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-18 04:43:30.714 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-18 04:43:30.714 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57309/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-18 04:43:30.717 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"L3dha3UvMi9ycy8yLzA=","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773809010385342715,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773809010598378049,"ephemeral":false,"proof":""}]'
2026-03-18 04:43:30.728 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-18 04:43:30.729 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-18 04:43:30.729 DEBUG [src.node.waku_node] Stopping container with id f2890a929d0a
2026-03-18 04:43:31.326 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:43:31.328 DEBUG [src.node.waku_node] Stopping container with id f16772bd829e
2026-03-18 04:43:31.950 DEBUG [src.node.waku_node] Container stopped.
2026-03-18 04:43:31.953 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-18 04:43:32.030 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:43:32.066 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-18 04:43:32.067 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-18 04:43:32.067 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-18 04:43:32.067 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-18_04-43-27__22b1607d-5728-402e-aca9-c870d9ef75df__wakuorg_nwaku:latest.log
2026-03-18 04:43:32.069 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-18_04-43-27__22b1607d-5728-402e-aca9-c870d9ef75df__wakuorg_nwaku:latest.log