257 lines
46 KiB
Plaintext

2026-03-11 04:36:24.292 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-11 04:36:24.292 DEBUG [tests.conftest] Running test: test_unsubscribe_from_all_content_topics_one_by_one with id: 2026-03-11_04-36-24__3ad3ce87-4ad9-41a5-807b-f7c1c9dab8f9
2026-03-11 04:36:24.293 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-11 04:36:24.293 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-11 04:36:24.293 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-11 04:36:24.301 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:36:24.301 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-11_04-36-24__3ad3ce87-4ad9-41a5-807b-f7c1c9dab8f9__wakuorg_nwaku:latest.log
2026-03-11 04:36:24.301 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:36:24.301 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:36:24.303 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:36:24.303 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.161.83
2026-03-11 04:36:24.303 DEBUG [src.node.docker_mananger] Generated ports ['50395', '50396', '50397', '50398', '50399']
2026-03-11 04:36:24.303 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:36:24.303 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:36:24.303 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:36:24.303 DEBUG [src.node.docker_mananger] docker run -i -t -p 50395:50395 -p 50396:50396 -p 50397:50397 -p 50398:50398 -p 50399:50399 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=50397 --rest-port=50395 --tcp-port=50396 --discv5-udp-port=50398 --rest-address=0.0.0.0 --nat=extip:172.18.161.83 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=abdfe4e1bb8adcd1695d1b24aaa7cde7de4cabc799af8a0e0fb94b49ddd52184 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=50399 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
2026-03-11 04:36:24.492 ERROR [src.node.docker_mananger] Max retries reached for container f5059985ffb8. Exiting log stream.
2026-03-11 04:36:24.496 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.161.83 waku ba5fe37d40ea7d9d449267387786e1c93b10f3e19c8f5a97878974f97408b08b
2026-03-11 04:36:24.531 DEBUG [src.node.docker_mananger] Container started with ID ba5fe37d40ea. Setting up logs at ./log/docker/node1_2026-03-11_04-36-24__3ad3ce87-4ad9-41a5-807b-f7c1c9dab8f9__wakuorg_nwaku:latest.log
2026-03-11 04:36:24.533 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 50395
2026-03-11 04:36:24.534 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:36:25.041 ERROR [src.node.docker_mananger] Max retries reached for container 26fe7e8333e2. Exiting log stream.
2026-03-11 04:36:25.535 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50395/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:25.538 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:25.638 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50395/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:25.641 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:25.742 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50395/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:25.744 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:25.845 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50395/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:25.847 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:25.948 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50395/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:25.950 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:26.051 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50395/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:26.053 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:26.154 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50395/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:26.156 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:26.257 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50395/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:26.259 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:36:26.260 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:36:26.260 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50395/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:26.262 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.161.83/tcp/50396/p2p/16Uiu2HAmU2N6HhuzMcKDQqgMQWTnLWwcAWuWqyHdEB7i8hNYDGYc","/ip4/172.18.161.83/tcp/50397/ws/p2p/16Uiu2HAmU2N6HhuzMcKDQqgMQWTnLWwcAWuWqyHdEB7i8hNYDGYc"],"enrUri":"enr:-L24QKMUBdfV_H_xITsTYHujuolTttAw1lp75F4XstM1w3IsKVi9hWceyu_gL9VUuL7Q3me9r87jeg3iYtSuaAzlGZ4CgmlkgnY0gmlwhKwSoVOKbXVsdGlhZGRyc5YACASsEqFTBsTcAAoErBKhUwbE3d0DgnJzhQACAQAAiXNlY3AyNTZrMaED5FkVe-NmDkr3KrUUNDOD8-qM4dJfGSBUxEgwyrVi2bWDdGNwgsTcg3VkcILE3oV3YWt1MgU"}'
2026-03-11 04:36:26.263 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:36:26.270 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:36:26.270 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-11_04-36-24__3ad3ce87-4ad9-41a5-807b-f7c1c9dab8f9__wakuorg_nwaku:latest.log
2026-03-11 04:36:26.270 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:36:26.270 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:36:26.271 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:36:26.271 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.124.109
2026-03-11 04:36:26.272 DEBUG [src.node.docker_mananger] Generated ports ['51064', '51065', '51066', '51067', '51068']
2026-03-11 04:36:26.272 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:36:26.272 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:36:26.272 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:36:26.272 DEBUG [src.node.docker_mananger] docker run -i -t -p 51064:51064 -p 51065:51065 -p 51066:51066 -p 51067:51067 -p 51068:51068 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=51066 --rest-port=51064 --tcp-port=51065 --discv5-udp-port=51067 --rest-address=0.0.0.0 --nat=extip:172.18.124.109 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=afbdcde9ce5bdbd4e67febdd926e4151ad9a51fea9ad729c04a0f90fa6a44545 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=51068 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QKMUBdfV_H_xITsTYHujuolTttAw1lp75F4XstM1w3IsKVi9hWceyu_gL9VUuL7Q3me9r87jeg3iYtSuaAzlGZ4CgmlkgnY0gmlwhKwSoVOKbXVsdGlhZGRyc5YACASsEqFTBsTcAAoErBKhUwbE3d0DgnJzhQACAQAAiXNlY3AyNTZrMaED5FkVe-NmDkr3KrUUNDOD8-qM4dJfGSBUxEgwyrVi2bWDdGNwgsTcg3VkcILE3oV3YWt1MgU --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
2026-03-11 04:36:26.468 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.124.109 waku 51b9d938864e29e2e4e71594b3d767c0678345a90f1a6dea0e55fbb8baa39f3e
2026-03-11 04:36:26.504 DEBUG [src.node.docker_mananger] Container started with ID 51b9d938864e. Setting up logs at ./log/docker/node2_2026-03-11_04-36-24__3ad3ce87-4ad9-41a5-807b-f7c1c9dab8f9__wakuorg_nwaku:latest.log
2026-03-11 04:36:26.505 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 51064
2026-03-11 04:36:26.505 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:36:27.505 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:51064/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:27.508 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:27.609 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:51064/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:27.611 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:27.712 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:51064/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:27.714 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:36:27.715 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:36:27.715 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:51064/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:27.717 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.124.109/tcp/51065/p2p/16Uiu2HAmTr2JotNTrmQD732nAdxj3cAPjhcFEMTp7b8DMB4o3H33","/ip4/172.18.124.109/tcp/51066/ws/p2p/16Uiu2HAmTr2JotNTrmQD732nAdxj3cAPjhcFEMTp7b8DMB4o3H33"],"enrUri":"enr:-L24QIMaybDCSBs-BL80GjpZ4zYQspgRY4JbwQQPsDD67NUbAUOzq3S0O9htKbV7FxNbI9N9on62bAgmgQLrdgWo0VMCgmlkgnY0gmlwhKwSfG2KbXVsdGlhZGRyc5YACASsEnxtBsd5AAoErBJ8bQbHet0DgnJzhQACAQAAiXNlY3AyNTZrMaED4bLn5Tyjfk9gtATIiYvNy5IlC-c3OfutYj1AzFSXm0aDdGNwgsd5g3VkcILHe4V3YWt1MgE"}'
2026-03-11 04:36:27.718 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:36:27.718 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:51064/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.161.83/tcp/50396/p2p/16Uiu2HAmU2N6HhuzMcKDQqgMQWTnLWwcAWuWqyHdEB7i8hNYDGYc"]'
2026-03-11 04:36:27.739 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:27.740 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:50395/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
2026-03-11 04:36:27.754 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:27.754 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:51064/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
2026-03-11 04:36:27.760 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:27.761 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:50395/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:36:27.776 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:27.777 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-11 04:36:27.877 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-11 04:36:27.878 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50395/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:27.880 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773203786650303784,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773203786665556970,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773203787761245442,"ephemeral":false,"proof":""}]'
2026-03-11 04:36:27.927 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-11 04:36:27.928 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-11 04:36:27.928 DEBUG [src.node.waku_node] Stopping container with id ba5fe37d40ea
2026-03-11 04:36:28.530 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:36:28.530 DEBUG [src.node.waku_node] Stopping container with id 51b9d938864e
2026-03-11 04:36:29.125 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:36:29.127 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-11 04:36:29.160 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:36:29.189 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:36:29.190 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-11 04:36:29.190 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-11 04:36:29.190 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-11_04-36-24__3ad3ce87-4ad9-41a5-807b-f7c1c9dab8f9__wakuorg_nwaku:latest.log
2026-03-11 04:36:29.192 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-11_04-36-24__3ad3ce87-4ad9-41a5-807b-f7c1c9dab8f9__wakuorg_nwaku:latest.log
2026-03-11 04:36:29.197 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-11 04:36:29.197 DEBUG [tests.conftest] Running test: test_unsubscribe_from_all_content_topics_one_by_one with id: 2026-03-11_04-36-29__717c4e26-9f20-45b2-b67e-834f418714a8
2026-03-11 04:36:29.197 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-11 04:36:29.198 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-11 04:36:29.198 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-11 04:36:29.206 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:36:29.206 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-11_04-36-29__717c4e26-9f20-45b2-b67e-834f418714a8__wakuorg_nwaku:latest.log
2026-03-11 04:36:29.206 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:36:29.206 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:36:29.208 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:36:29.208 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.247.239
2026-03-11 04:36:29.208 DEBUG [src.node.docker_mananger] Generated ports ['53698', '53699', '53700', '53701', '53702']
2026-03-11 04:36:29.208 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:36:29.208 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:36:29.208 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:36:29.209 DEBUG [src.node.docker_mananger] docker run -i -t -p 53698:53698 -p 53699:53699 -p 53700:53700 -p 53701:53701 -p 53702:53702 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=53700 --rest-port=53698 --tcp-port=53699 --discv5-udp-port=53701 --rest-address=0.0.0.0 --nat=extip:172.18.247.239 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=2aee26d741add104e975a9bcf166a0b2aff3ae2e11992bf89ed3954cab0a6b8f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=53702 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
2026-03-11 04:36:29.395 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.247.239 waku 86db23feb757f4744c1a3c348e2dce1f2920dcdf53d6a66c7bd450c58f1b248b
2026-03-11 04:36:29.396 ERROR [src.node.docker_mananger] Max retries reached for container ba5fe37d40ea. Exiting log stream.
2026-03-11 04:36:29.434 DEBUG [src.node.docker_mananger] Container started with ID 86db23feb757. Setting up logs at ./log/docker/node1_2026-03-11_04-36-29__717c4e26-9f20-45b2-b67e-834f418714a8__wakuorg_nwaku:latest.log
2026-03-11 04:36:29.435 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 53698
2026-03-11 04:36:29.435 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:36:29.941 ERROR [src.node.docker_mananger] Max retries reached for container 51b9d938864e. Exiting log stream.
2026-03-11 04:36:30.438 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53698/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:30.441 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:36:30.442 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:36:30.442 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53698/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:30.444 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.247.239/tcp/53699/p2p/16Uiu2HAmNXhyCGm5ThpyTx9ZWfFkMaemwWVpmC1tgMDsnUs1tiCq","/ip4/172.18.247.239/tcp/53700/ws/p2p/16Uiu2HAmNXhyCGm5ThpyTx9ZWfFkMaemwWVpmC1tgMDsnUs1tiCq"],"enrUri":"enr:-L24QMR76aWhdsjwWJmr8oGB-AlaKnxn9-fcbJEJPNtUaJkOcMS858T8oCbH-X3YxO8b9-k-CB6F4jQASs_GM3NcvcwCgmlkgnY0gmlwhKwS9--KbXVsdGlhZGRyc5YACASsEvfvBtHDAAoErBL37wbRxN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDkrdIF3mbcTPH4xi168VlDRXTxq_k_k0mPg3QgjhZb-qDdGNwgtHDg3VkcILRxYV3YWt1MgU"}'
2026-03-11 04:36:30.444 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:36:30.452 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:36:30.452 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-11_04-36-29__717c4e26-9f20-45b2-b67e-834f418714a8__wakuorg_nwaku:latest.log
2026-03-11 04:36:30.452 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:36:30.452 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:36:30.453 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:36:30.454 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.188.31
2026-03-11 04:36:30.454 DEBUG [src.node.docker_mananger] Generated ports ['61922', '61923', '61924', '61925', '61926']
2026-03-11 04:36:30.454 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:36:30.454 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:36:30.454 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:36:30.454 DEBUG [src.node.docker_mananger] docker run -i -t -p 61922:61922 -p 61923:61923 -p 61924:61924 -p 61925:61925 -p 61926:61926 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=61924 --rest-port=61922 --tcp-port=61923 --discv5-udp-port=61925 --rest-address=0.0.0.0 --nat=extip:172.18.188.31 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=adb4ffcf58aca8b5078a30eed1b21357e46ea0aed10bcbb0c2aceabdd70dff6d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=61926 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QMR76aWhdsjwWJmr8oGB-AlaKnxn9-fcbJEJPNtUaJkOcMS858T8oCbH-X3YxO8b9-k-CB6F4jQASs_GM3NcvcwCgmlkgnY0gmlwhKwS9--KbXVsdGlhZGRyc5YACASsEvfvBtHDAAoErBL37wbRxN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDkrdIF3mbcTPH4xi168VlDRXTxq_k_k0mPg3QgjhZb-qDdGNwgtHDg3VkcILRxYV3YWt1MgU --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
2026-03-11 04:36:30.677 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.188.31 waku 8617b1e9aa676c4f6998bb449f55d5ac92ab05b8816ba4cc431628eab4f2bb2a
2026-03-11 04:36:30.716 DEBUG [src.node.docker_mananger] Container started with ID 8617b1e9aa67. Setting up logs at ./log/docker/node2_2026-03-11_04-36-29__717c4e26-9f20-45b2-b67e-834f418714a8__wakuorg_nwaku:latest.log
2026-03-11 04:36:30.716 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 61922
2026-03-11 04:36:30.717 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:36:31.717 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:61922/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:31.751 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:31.853 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:61922/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:31.857 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:36:31.858 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:36:31.858 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:61922/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:31.861 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.188.31/tcp/61923/p2p/16Uiu2HAkwNmW5oPXRrm3UH56ZXSNNM8RszRnKf1h7ShothyGqtgo","/ip4/172.18.188.31/tcp/61924/ws/p2p/16Uiu2HAkwNmW5oPXRrm3UH56ZXSNNM8RszRnKf1h7ShothyGqtgo"],"enrUri":"enr:-L24QGWWfgcXNx6D9tleQXWli5bnpFFpZMtvB-xYd7XIBDLyCRX-HCY9FwOhnX11WUo8H9Kp4nmbJcrvvtqihwI1cLECgmlkgnY0gmlwhKwSvB-KbXVsdGlhZGRyc5YACASsErwfBvHjAAoErBK8Hwbx5N0DgnJzhQACAQAAiXNlY3AyNTZrMaECHPjgHSYcxT3KN5zOrTmlv2ltFy17qvuqxWeTcfjvUSCDdGNwgvHjg3VkcILx5YV3YWt1MgE"}'
2026-03-11 04:36:31.862 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:36:31.862 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:61922/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.247.239/tcp/53699/p2p/16Uiu2HAmNXhyCGm5ThpyTx9ZWfFkMaemwWVpmC1tgMDsnUs1tiCq"]'
2026-03-11 04:36:31.899 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:31.900 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:53698/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
2026-03-11 04:36:31.906 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:31.907 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:61922/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
2026-03-11 04:36:31.913 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:31.915 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:53698/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:36:31.925 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:31.926 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-11 04:36:32.026 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-11 04:36:32.027 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53698/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:32.030 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773203790528011979,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773203791593745034,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773203791915255671,"ephemeral":false,"proof":""}]'
2026-03-11 04:36:32.041 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-11 04:36:32.042 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-11 04:36:32.042 DEBUG [src.node.waku_node] Stopping container with id 86db23feb757
2026-03-11 04:36:32.692 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:36:32.694 DEBUG [src.node.waku_node] Stopping container with id 8617b1e9aa67
2026-03-11 04:36:33.325 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:36:33.326 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-11 04:36:33.367 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:36:33.409 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:36:33.410 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-11 04:36:33.411 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-11 04:36:33.411 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-11_04-36-29__717c4e26-9f20-45b2-b67e-834f418714a8__wakuorg_nwaku:latest.log
2026-03-11 04:36:33.413 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-11_04-36-29__717c4e26-9f20-45b2-b67e-834f418714a8__wakuorg_nwaku:latest.log
2026-03-11 04:36:33.417 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-11 04:36:33.417 DEBUG [tests.conftest] Running test: test_unsubscribe_from_all_content_topics_one_by_one with id: 2026-03-11_04-36-33__1caacb4e-e64f-4019-ac18-287f337064fc
2026-03-11 04:36:33.418 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-11 04:36:33.418 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-11 04:36:33.418 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-11 04:36:33.426 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:36:33.426 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-11_04-36-33__1caacb4e-e64f-4019-ac18-287f337064fc__wakuorg_nwaku:latest.log
2026-03-11 04:36:33.427 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:36:33.427 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:36:33.428 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:36:33.428 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.60.104
2026-03-11 04:36:33.429 DEBUG [src.node.docker_mananger] Generated ports ['5254', '5255', '5256', '5257', '5258']
2026-03-11 04:36:33.429 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:36:33.429 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:36:33.429 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:36:33.429 DEBUG [src.node.docker_mananger] docker run -i -t -p 5254:5254 -p 5255:5255 -p 5256:5256 -p 5257:5257 -p 5258:5258 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=5256 --rest-port=5254 --tcp-port=5255 --discv5-udp-port=5257 --rest-address=0.0.0.0 --nat=extip:172.18.60.104 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=efe3d6d0cbf6770773f2db100ee2d4bebc52c2cdd3de7df451f59abc089bfbeb --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=5258 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
2026-03-11 04:36:33.505 ERROR [src.node.docker_mananger] Max retries reached for container 86db23feb757. Exiting log stream.
2026-03-11 04:36:33.624 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.60.104 waku 0115466ac7fa33a56bd069641ee9bf91d75a921419df83c9bdf2309504c70210
2026-03-11 04:36:33.665 DEBUG [src.node.docker_mananger] Container started with ID 0115466ac7fa. Setting up logs at ./log/docker/node1_2026-03-11_04-36-33__1caacb4e-e64f-4019-ac18-287f337064fc__wakuorg_nwaku:latest.log
2026-03-11 04:36:33.666 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 5254
2026-03-11 04:36:33.668 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:36:34.222 ERROR [src.node.docker_mananger] Max retries reached for container 8617b1e9aa67. Exiting log stream.
2026-03-11 04:36:34.668 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5254/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:34.671 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:34.771 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5254/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:34.776 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:34.876 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5254/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:34.879 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:34.980 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5254/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:35.002 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:35.103 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5254/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:35.106 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:36:35.106 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:36:35.106 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5254/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:35.109 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.60.104/tcp/5255/p2p/16Uiu2HAmBcSzjGrFahki6zo4yFixtFCSggfvgoUi4t2nTXkQTTyM","/ip4/172.18.60.104/tcp/5256/ws/p2p/16Uiu2HAmBcSzjGrFahki6zo4yFixtFCSggfvgoUi4t2nTXkQTTyM"],"enrUri":"enr:-L24QLaLY53x_Xqlj4TsdT4xaBfUYRAik2Pe1HA_Om_n6UYNJrRI42jAfM8k6VRUYmnjzNQRv-pUEmY5bnWPfesGkOcCgmlkgnY0gmlwhKwSPGiKbXVsdGlhZGRyc5YACASsEjxoBhSHAAoErBI8aAYUiN0DgnJzhQACAQAAiXNlY3AyNTZrMaEC8H2y5i075j6E8WpxJQy436i4t-JSE0hT8eTKIWfrgEyDdGNwghSHg3VkcIIUiYV3YWt1MgU"}'
2026-03-11 04:36:35.109 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:36:35.116 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:36:35.116 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-11_04-36-33__1caacb4e-e64f-4019-ac18-287f337064fc__wakuorg_nwaku:latest.log
2026-03-11 04:36:35.116 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:36:35.117 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:36:35.118 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:36:35.118 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.204.162
2026-03-11 04:36:35.118 DEBUG [src.node.docker_mananger] Generated ports ['3296', '3297', '3298', '3299', '3300']
2026-03-11 04:36:35.119 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:36:35.119 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:36:35.119 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:36:35.119 DEBUG [src.node.docker_mananger] docker run -i -t -p 3296:3296 -p 3297:3297 -p 3298:3298 -p 3299:3299 -p 3300:3300 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=3298 --rest-port=3296 --tcp-port=3297 --discv5-udp-port=3299 --rest-address=0.0.0.0 --nat=extip:172.18.204.162 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=de7f8548a1f3963cfe9d7bfbafbdeff76a1feba4ef9fc87d09640aaffeca3aca --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=3300 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QLaLY53x_Xqlj4TsdT4xaBfUYRAik2Pe1HA_Om_n6UYNJrRI42jAfM8k6VRUYmnjzNQRv-pUEmY5bnWPfesGkOcCgmlkgnY0gmlwhKwSPGiKbXVsdGlhZGRyc5YACASsEjxoBhSHAAoErBI8aAYUiN0DgnJzhQACAQAAiXNlY3AyNTZrMaEC8H2y5i075j6E8WpxJQy436i4t-JSE0hT8eTKIWfrgEyDdGNwghSHg3VkcIIUiYV3YWt1MgU --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
2026-03-11 04:36:35.321 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.204.162 waku d89c3f9b3a1963d7042acbe132c27728196c36bdf7fcf3ee52c635320d7dca87
2026-03-11 04:36:35.358 DEBUG [src.node.docker_mananger] Container started with ID d89c3f9b3a19. Setting up logs at ./log/docker/node2_2026-03-11_04-36-33__1caacb4e-e64f-4019-ac18-287f337064fc__wakuorg_nwaku:latest.log
2026-03-11 04:36:35.358 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 3296
2026-03-11 04:36:35.358 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:36:36.359 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:3296/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:36.361 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:36.462 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:3296/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:36.465 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:36.565 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:3296/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:36.586 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:36.686 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:3296/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:36.689 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:36:36.689 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:36:36.689 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:3296/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:36.691 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.204.162/tcp/3297/p2p/16Uiu2HAm1d2Dq6kzXsF6r9pmAQEBvHTjkucDpayRoPak6mvP5RX2","/ip4/172.18.204.162/tcp/3298/ws/p2p/16Uiu2HAm1d2Dq6kzXsF6r9pmAQEBvHTjkucDpayRoPak6mvP5RX2"],"enrUri":"enr:-L24QKmavwirVJNYwl308Y6UUaZdDTOieAHTchpWssO6j8bxbiCtC1ASO8L_PaO6KFGuv7JoNwbX_NTcGGcm9AXF2TICgmlkgnY0gmlwhKwSzKKKbXVsdGlhZGRyc5YACASsEsyiBgzhAAoErBLMogYM4t0DgnJzhQACAQAAiXNlY3AyNTZrMaECXA5l7pja2qBuOCvl3GsZPEjnz6nFxgUCqUPKMT_4WM2DdGNwggzhg3VkcIIM44V3YWt1MgE"}'
2026-03-11 04:36:36.692 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:36:36.692 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:3296/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.60.104/tcp/5255/p2p/16Uiu2HAmBcSzjGrFahki6zo4yFixtFCSggfvgoUi4t2nTXkQTTyM"]'
2026-03-11 04:36:36.719 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:36.721 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:5254/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
2026-03-11 04:36:36.728 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:36.729 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:3296/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
2026-03-11 04:36:36.733 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:36.734 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:5254/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:36:36.745 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:36.745 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-11 04:36:36.846 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-11 04:36:36.847 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5254/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:36.853 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773203796734588723,"ephemeral":false,"proof":""},{"payload":"L215YXBwLzEvbGF0ZXN0L3Byb3Rv","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773203796802946580,"ephemeral":false,"proof":""}]'
2026-03-11 04:36:36.875 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-11 04:36:36.879 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-11 04:36:36.879 DEBUG [src.node.waku_node] Stopping container with id 0115466ac7fa
2026-03-11 04:36:37.483 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:36:37.485 DEBUG [src.node.waku_node] Stopping container with id d89c3f9b3a19
2026-03-11 04:36:38.049 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:36:38.052 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-11 04:36:38.085 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:36:38.118 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:36:38.120 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-11 04:36:38.120 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-11 04:36:38.121 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-11_04-36-33__1caacb4e-e64f-4019-ac18-287f337064fc__wakuorg_nwaku:latest.log
2026-03-11 04:36:38.123 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-11_04-36-33__1caacb4e-e64f-4019-ac18-287f337064fc__wakuorg_nwaku:latest.log