210 lines
39 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_publish_on_not_subscribed_content_topic_works with id: 2026-03-03_04-35-08__bd6a24c1-596b-4d26-bf38-dbcd1fb0a0f6
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-03_04-35-08__bd6a24c1-596b-4d26-bf38-dbcd1fb0a0f6__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.98.201
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['20448', '20449', '20450', '20451', '20452']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 20448:20448 -p 20449:20449 -p 20450:20450 -p 20451:20451 -p 20452:20452 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=20450 --rest-port=20448 --tcp-port=20449 --discv5-udp-port=20451 --rest-address=0.0.0.0 --nat=extip:172.18.98.201 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=bb595f65ee8bad1ace23b9ccf5c8af9ab7ddbcaef72db7fbfeceefcf24716681 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=20452 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.98.201 waku 021a9c2bfefb717a633dad9d54d6506f75f89998142f2b6ad47d7048bd6096e4
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 021a9c2bfefb. Setting up logs at ./log/docker/node1_2026-03-03_04-35-08__bd6a24c1-596b-4d26-bf38-dbcd1fb0a0f6__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 20448
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 76f50ce91dfe. Exiting log stream.
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 43958fd18b8f. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:20448/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:20448/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.98.201/tcp/20449/p2p/16Uiu2HAmTeB4gq3L38hswgpBif7HtsPhGDtZjVoL7TPJGrfqdXiW","/ip4/172.18.98.201/tcp/20450/ws/p2p/16Uiu2HAmTeB4gq3L38hswgpBif7HtsPhGDtZjVoL7TPJGrfqdXiW"],"enrUri":"enr:-L24QF0Q70Ny_f9i8mCpQ2x66uHAOxHRAt5GMuIj2fj8YwWBfCoKNYFItDkMHCoJ5XqD0kH55gmYe8xiBHpbqbWC32oCgmlkgnY0gmlwhKwSYsmKbXVsdGlhZGRyc5YACASsEmLJBk_hAAoErBJiyQZP4t0DgnJzhQACAQAAiXNlY3AyNTZrMaED3qnWUGsN3cnzv_1WcqEs9u6BnVVnrM4XtzLA6WraLr-DdGNwgk_hg3VkcIJP44V3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node2_2026-03-03_04-35-08__bd6a24c1-596b-4d26-bf38-dbcd1fb0a0f6__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.1.100
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['25799', '25800', '25801', '25802', '25803']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 25799:25799 -p 25800:25800 -p 25801:25801 -p 25802:25802 -p 25803:25803 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=25801 --rest-port=25799 --tcp-port=25800 --discv5-udp-port=25802 --rest-address=0.0.0.0 --nat=extip:172.18.1.100 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=dc3e9c35b7d60e31fdc9caade9365a7eba0d6cdc48e7fbb8c96cabc9ea8331ab --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=25803 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QF0Q70Ny_f9i8mCpQ2x66uHAOxHRAt5GMuIj2fj8YwWBfCoKNYFItDkMHCoJ5XqD0kH55gmYe8xiBHpbqbWC32oCgmlkgnY0gmlwhKwSYsmKbXVsdGlhZGRyc5YACASsEmLJBk_hAAoErBJiyQZP4t0DgnJzhQACAQAAiXNlY3AyNTZrMaED3qnWUGsN3cnzv_1WcqEs9u6BnVVnrM4XtzLA6WraLr-DdGNwgk_hg3VkcIJP44V3YWt1MgU --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.1.100 waku 24dd9a2e8116b9014069a4b591d0ffe0470ca80499435715b1a5fb06c0d797af
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 24dd9a2e8116. Setting up logs at ./log/docker/node2_2026-03-03_04-35-08__bd6a24c1-596b-4d26-bf38-dbcd1fb0a0f6__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 25799
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:25799/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:25799/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.1.100/tcp/25800/p2p/16Uiu2HAmTtZZCwUUbw4y7Exe6LnTvkMoZnPXbksLRqJBXQqg8zqK","/ip4/172.18.1.100/tcp/25801/ws/p2p/16Uiu2HAmTtZZCwUUbw4y7Exe6LnTvkMoZnPXbksLRqJBXQqg8zqK"],"enrUri":"enr:-L24QEDcOe2wIz8cYuIOAn0NTGCKeNuzJgaP6conLAKcp0MNZVnidTd_a8yG6FUv0u49Rqb-YJWUStxKxY9j2w8XTY0CgmlkgnY0gmlwhKwSAWSKbXVsdGlhZGRyc5YACASsEgFkBmTIAAoErBIBZAZkyd0DgnJzhQACAQAAiXNlY3AyNTZrMaED4llmYwm1fjDSwLsxN7npT3b3Fmo81YS3U8nu4NHxuT6DdGNwgmTIg3VkcIJkyoV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:25799/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.98.201/tcp/20449/p2p/16Uiu2HAmTeB4gq3L38hswgpBif7HtsPhGDtZjVoL7TPJGrfqdXiW"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:20448/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:25799/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:20448/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/toychat/2/huilong/proto", "timestamp": '$(date +%s%N)'}'
ERROR  src.node.api_clients.base_client:base_client.py:16 HTTP error occurred: 400 Client Error: Bad Request for url: http://127.0.0.1:20448/relay/v1/auto/messages. Response content: b'Failed to publish: publish failed in relay: NoPeersToPublish'
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 021a9c2bfefb
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 24dd9a2e8116
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:73 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:74 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-03-03_04-35-08__bd6a24c1-596b-4d26-bf38-dbcd1fb0a0f6__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-03-03_04-35-08__bd6a24c1-596b-4d26-bf38-dbcd1fb0a0f6__wakuorg_nwaku:latest.log
DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_publish_on_not_subscribed_content_topic_works with id: 2026-03-03_04-35-12__2d128cbd-9243-4768-ad2f-6ec8c70b5ed3
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-03_04-35-12__2d128cbd-9243-4768-ad2f-6ec8c70b5ed3__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.77.190
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['52645', '52646', '52647', '52648', '52649']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 52645:52645 -p 52646:52646 -p 52647:52647 -p 52648:52648 -p 52649:52649 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=52647 --rest-port=52645 --tcp-port=52646 --discv5-udp-port=52648 --rest-address=0.0.0.0 --nat=extip:172.18.77.190 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=70a9f9f8b9c42a982bab1fdac5e6d7aad3aa21fffab7058c6adb7c7ef0cc9cb6 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=52649 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.77.190 waku 486c2472b98c57c79953827e55569612ceb291008417f3eb6dce13a33f8af6e3
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 021a9c2bfefb. Exiting log stream.
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 486c2472b98c. Setting up logs at ./log/docker/node1_2026-03-03_04-35-12__2d128cbd-9243-4768-ad2f-6ec8c70b5ed3__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 52645
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 24dd9a2e8116. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:52645/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:52645/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.77.190/tcp/52646/p2p/16Uiu2HAkwSJ9QfuxKrwYC98fgayWjfkkNUJpPaunWgxn1faLeJQm","/ip4/172.18.77.190/tcp/52647/ws/p2p/16Uiu2HAkwSJ9QfuxKrwYC98fgayWjfkkNUJpPaunWgxn1faLeJQm"],"enrUri":"enr:-L24QCio8FZu5kju8yoyLgsj-9Shw6muGqB3VjlFoflWGKR6fFU-QIWqT7iY7c65QkOVygqWfHUJSRHMJ1TYXoh1Ws0CgmlkgnY0gmlwhKwSTb6KbXVsdGlhZGRyc5YACASsEk2-Bs2mAAoErBJNvgbNp90DgnJzhQACAQAAiXNlY3AyNTZrMaECHeBEQiA3jmlnyA_X_PQzBW9m0yjLRZ0tZUHkWiyCEt6DdGNwgs2mg3VkcILNqIV3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node2_2026-03-03_04-35-12__2d128cbd-9243-4768-ad2f-6ec8c70b5ed3__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.142.137
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['53654', '53655', '53656', '53657', '53658']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 53654:53654 -p 53655:53655 -p 53656:53656 -p 53657:53657 -p 53658:53658 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=53656 --rest-port=53654 --tcp-port=53655 --discv5-udp-port=53657 --rest-address=0.0.0.0 --nat=extip:172.18.142.137 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=c50704ddd27b57c8dde9cbd480c2ce338ffeb03bb171b318daa037e9e01ff91c --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=53658 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QCio8FZu5kju8yoyLgsj-9Shw6muGqB3VjlFoflWGKR6fFU-QIWqT7iY7c65QkOVygqWfHUJSRHMJ1TYXoh1Ws0CgmlkgnY0gmlwhKwSTb6KbXVsdGlhZGRyc5YACASsEk2-Bs2mAAoErBJNvgbNp90DgnJzhQACAQAAiXNlY3AyNTZrMaECHeBEQiA3jmlnyA_X_PQzBW9m0yjLRZ0tZUHkWiyCEt6DdGNwgs2mg3VkcILNqIV3YWt1MgU --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.142.137 waku 149577551d33b505108e2550b4ebeced6f1a5c80a288e1fcab63fad38c1a4a82
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 149577551d33. Setting up logs at ./log/docker/node2_2026-03-03_04-35-12__2d128cbd-9243-4768-ad2f-6ec8c70b5ed3__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 53654
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:53654/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:53654/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.142.137/tcp/53655/p2p/16Uiu2HAmLPjozjRaqsSKMUoFPZkhfXvZS2qUD7hDUKj7TgsgjK4i","/ip4/172.18.142.137/tcp/53656/ws/p2p/16Uiu2HAmLPjozjRaqsSKMUoFPZkhfXvZS2qUD7hDUKj7TgsgjK4i"],"enrUri":"enr:-L24QCs3H4YnTmb_OMDm-8And52uQ1wuq9Jqg26dd2McyeZwd2btk_5K1WheVGgITm4VdLT4I7KPoRKdIRZ-c-DZEyACgmlkgnY0gmlwhKwSjomKbXVsdGlhZGRyc5YACASsEo6JBtGXAAoErBKOiQbRmN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDcvVX5kHLmkXbIxCyFqr5cRTiUTiFTkYwIWQlzceUvV-DdGNwgtGXg3VkcILRmYV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:53654/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.77.190/tcp/52646/p2p/16Uiu2HAkwSJ9QfuxKrwYC98fgayWjfkkNUJpPaunWgxn1faLeJQm"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:52645/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:53654/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:52645/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/toychat/2/huilong/proto", "timestamp": '$(date +%s%N)'}'
ERROR  src.node.api_clients.base_client:base_client.py:16 HTTP error occurred: 400 Client Error: Bad Request for url: http://127.0.0.1:52645/relay/v1/auto/messages. Response content: b'Failed to publish: publish failed in relay: NoPeersToPublish'
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 486c2472b98c
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 149577551d33
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:73 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:74 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-03-03_04-35-12__2d128cbd-9243-4768-ad2f-6ec8c70b5ed3__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-03-03_04-35-12__2d128cbd-9243-4768-ad2f-6ec8c70b5ed3__wakuorg_nwaku:latest.log
DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_publish_on_not_subscribed_content_topic_works with id: 2026-03-03_04-35-15__26da83fe-6f6c-42bc-9c6a-52a7483e62bb
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-03_04-35-15__26da83fe-6f6c-42bc-9c6a-52a7483e62bb__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.43.34
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['49278', '49279', '49280', '49281', '49282']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 49278:49278 -p 49279:49279 -p 49280:49280 -p 49281:49281 -p 49282:49282 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=49280 --rest-port=49278 --tcp-port=49279 --discv5-udp-port=49281 --rest-address=0.0.0.0 --nat=extip:172.18.43.34 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=ba758f81fa4dc247e4544166a63e8cff1bcac3b8a7dfcd6c7a97abb6b0e2362a --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=49282 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.43.34 waku c97d14fee6b20d6dd5c8ed9baa2ab8e0135507a914aed39af9c195c4bc72a7aa
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 486c2472b98c. Exiting log stream.
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID c97d14fee6b2. Setting up logs at ./log/docker/node1_2026-03-03_04-35-15__26da83fe-6f6c-42bc-9c6a-52a7483e62bb__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 49278
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 149577551d33. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:49278/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:49278/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.43.34/tcp/49279/p2p/16Uiu2HAmVEedXkGRTuvYCZnziwkZ8QUx3BHRoSdKvyEnDxBUUiaZ","/ip4/172.18.43.34/tcp/49280/ws/p2p/16Uiu2HAmVEedXkGRTuvYCZnziwkZ8QUx3BHRoSdKvyEnDxBUUiaZ"],"enrUri":"enr:-L24QE2iOxK0RiEWKKL4sqvw49iSiqj4jopUCQjovkfg2TwGW2C8N3AXavDK0k_RDwSuEAaUs4-tUE08zgb9u_R6NvUCgmlkgnY0gmlwhKwSKyKKbXVsdGlhZGRyc5YACASsEisiBsB_AAoErBIrIgbAgN0DgnJzhQACAQAAiXNlY3AyNTZrMaED9lpw3TZ-6SGK305OPpvNkeEt2tzE-vLmSZzKEYZLmkaDdGNwgsB_g3VkcILAgYV3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node2_2026-03-03_04-35-15__26da83fe-6f6c-42bc-9c6a-52a7483e62bb__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.36.60
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['18655', '18656', '18657', '18658', '18659']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 18655:18655 -p 18656:18656 -p 18657:18657 -p 18658:18658 -p 18659:18659 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=18657 --rest-port=18655 --tcp-port=18656 --discv5-udp-port=18658 --rest-address=0.0.0.0 --nat=extip:172.18.36.60 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=df83f1fdd5dbb0d8bb6dfc7adfbce248838d1fe4cd4d8b7d4a9fafb7df9ec588 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=18659 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QE2iOxK0RiEWKKL4sqvw49iSiqj4jopUCQjovkfg2TwGW2C8N3AXavDK0k_RDwSuEAaUs4-tUE08zgb9u_R6NvUCgmlkgnY0gmlwhKwSKyKKbXVsdGlhZGRyc5YACASsEisiBsB_AAoErBIrIgbAgN0DgnJzhQACAQAAiXNlY3AyNTZrMaED9lpw3TZ-6SGK305OPpvNkeEt2tzE-vLmSZzKEYZLmkaDdGNwgsB_g3VkcILAgYV3YWt1MgU --content-topic=/myapp/1/latest/proto --num-shards-in-network=8
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.36.60 waku e54423b9cc06af2a46e2ed4afceb613b553c68cb92043fe25c11508b17075091
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID e54423b9cc06. Setting up logs at ./log/docker/node2_2026-03-03_04-35-15__26da83fe-6f6c-42bc-9c6a-52a7483e62bb__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 18655
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:18655/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:18655/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.36.60/tcp/18656/p2p/16Uiu2HAmSXWgTRg9NpwrDHcsN2eDh2KRaXBfZMytS7H5q9ym977t","/ip4/172.18.36.60/tcp/18657/ws/p2p/16Uiu2HAmSXWgTRg9NpwrDHcsN2eDh2KRaXBfZMytS7H5q9ym977t"],"enrUri":"enr:-L24QJNbLXfiB1L7hOJZlYzJ-laGz0dPD_6227L1de-UKUckW9c-tqbBetigig4M4yQ_KdWYJ9Y6ZeyyLGjS2pjiInMCgmlkgnY0gmlwhKwSJDyKbXVsdGlhZGRyc5YACASsEiQ8BkjgAAoErBIkPAZI4d0DgnJzhQACAQAAiXNlY3AyNTZrMaEDzhlD57RPuOJBkSMIBxqqX7E4Leo4eEG9DCZbz2GL2WeDdGNwgkjgg3VkcIJI4oV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:18655/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.43.34/tcp/49279/p2p/16Uiu2HAmVEedXkGRTuvYCZnziwkZ8QUx3BHRoSdKvyEnDxBUUiaZ"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:49278/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:18655/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:49278/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/toychat/2/huilong/proto", "timestamp": '$(date +%s%N)'}'
ERROR  src.node.api_clients.base_client:base_client.py:16 HTTP error occurred: 400 Client Error: Bad Request for url: http://127.0.0.1:49278/relay/v1/auto/messages. Response content: b'Failed to publish: publish failed in relay: NoPeersToPublish'
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id c97d14fee6b2
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id e54423b9cc06
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:73 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:74 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-03-03_04-35-15__26da83fe-6f6c-42bc-9c6a-52a7483e62bb__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-03-03_04-35-15__26da83fe-6f6c-42bc-9c6a-52a7483e62bb__wakuorg_nwaku:latest.log