296 lines
50 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_cant_publish_on_not_subscribed_shard with id: 2026-03-20_04-41-18__27dc2e5c-e023-4d70-a3aa-7f2b251a692e
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-20_04-41-18__27dc2e5c-e023-4d70-a3aa-7f2b251a692e__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.66.222
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['3741', '3742', '3743', '3744', '3745']
DEBUG  src.node.waku_node:waku_node.py:72 Cluster id was resolved to: 2
DEBUG  src.node.waku_node:waku_node.py:163 Removing pubsub-topic from nwaku args
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 3741:3741 -p 3742:3742 -p 3743:3743 -p 3744:3744 -p 3745:3745 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=3743 --rest-port=3741 --tcp-port=3742 --discv5-udp-port=3744 --rest-address=0.0.0.0 --nat=extip:172.18.66.222 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=b2f7fb86bcdca8f4dcc7ded44f0f2e5efd2efefb0eeea1addfb92e0ebd5dacef --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=3745 --metrics-logging=true --relay=true --filter=true
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container fbd5a63287f6. Exiting log stream.
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.66.222 waku 81850c7894f0ea08c82717b1ca9020be87f80ae5b0192da32c38b75d6002769a
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 81850c7894f0. Setting up logs at ./log/docker/node1_2026-03-20_04-41-18__27dc2e5c-e023-4d70-a3aa-7f2b251a692e__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 3741
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 8aa703195b73. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:3741/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:3741/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:3741/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.66.222/tcp/3742/p2p/16Uiu2HAm1s5q2vnJMd1MtbV5Wgy2gRPXsgMRXAAPXVAW7iPdN5a8","/ip4/172.18.66.222/tcp/3743/ws/p2p/16Uiu2HAm1s5q2vnJMd1MtbV5Wgy2gRPXsgMRXAAPXVAW7iPdN5a8"],"enrUri":"enr:-L24QFl7WH9UztmnaAgtPaqXjxmAmTKvi8B-Z37OBLGTj2fxLz6I7kSsvgSv7P6FCqDDGdeQuiPr47fYFRHNlHd05pUCgmlkgnY0gmlwhKwSQt6KbXVsdGlhZGRyc5YACASsEkLeBg6eAAoErBJC3gYOn90DgnJzhQACAQAAiXNlY3AyNTZrMaECX6ibn8O2VT-UF655EdnQDFjEDG855hDzUqBsKuwkOZmDdGNwgg6eg3VkcIIOoIV3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node2_2026-03-20_04-41-18__27dc2e5c-e023-4d70-a3aa-7f2b251a692e__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.215.166
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['43504', '43505', '43506', '43507', '43508']
DEBUG  src.node.waku_node:waku_node.py:72 Cluster id was resolved to: 2
DEBUG  src.node.waku_node:waku_node.py:163 Removing pubsub-topic from nwaku args
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 43504:43504 -p 43505:43505 -p 43506:43506 -p 43507:43507 -p 43508:43508 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=43506 --rest-port=43504 --tcp-port=43505 --discv5-udp-port=43507 --rest-address=0.0.0.0 --nat=extip:172.18.215.166 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=f0cef9fd5baefe7a360d79ff2be5de9e42044a3ff6d0c7db1dbb5fffafc7dd98 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=43508 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QFl7WH9UztmnaAgtPaqXjxmAmTKvi8B-Z37OBLGTj2fxLz6I7kSsvgSv7P6FCqDDGdeQuiPr47fYFRHNlHd05pUCgmlkgnY0gmlwhKwSQt6KbXVsdGlhZGRyc5YACASsEkLeBg6eAAoErBJC3gYOn90DgnJzhQACAQAAiXNlY3AyNTZrMaECX6ibn8O2VT-UF655EdnQDFjEDG855hDzUqBsKuwkOZmDdGNwgg6eg3VkcIIOoIV3YWt1MgU
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.215.166 waku 2683e90be550a109d5a3b82e99fea6a6efab7b5090415d27e90c6580afc16201
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 2683e90be550. Setting up logs at ./log/docker/node2_2026-03-20_04-41-18__27dc2e5c-e023-4d70-a3aa-7f2b251a692e__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 43504
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43504/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43504/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43504/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43504/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43504/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43504/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43504/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43504/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43504/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43504/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43504/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43504/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.215.166/tcp/43505/p2p/16Uiu2HAmDtW7qxD2tJv8NqACXtCoqSMTDz3my1CdSbSqpPM5LhWn","/ip4/172.18.215.166/tcp/43506/ws/p2p/16Uiu2HAmDtW7qxD2tJv8NqACXtCoqSMTDz3my1CdSbSqpPM5LhWn"],"enrUri":"enr:-L24QBxZNOaXZN34qDI3KIxLpR3JIuLeawxMESoQU86wdxzPPc0IYdG6KNsegW5WIFPDzIxbyvrRE74RcWZXv5PyJbQCgmlkgnY0gmlwhKwS16aKbXVsdGlhZGRyc5YACASsEtemBqnxAAoErBLXpgap8t0DgnJzhQACAQAAiXNlY3AyNTZrMaEDElHlfUDf1TybPXv43Vu6R4u7EInAIfvmUUnKVj-NCxeDdGNwgqnxg3VkcIKp84V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:43504/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.66.222/tcp/3742/p2p/16Uiu2HAm1s5q2vnJMd1MtbV5Wgy2gRPXsgMRXAAPXVAW7iPdN5a8"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:3741/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:43504/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:3741/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.libs.common:common.py:47 Sleeping for 0.1 seconds
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:3741/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
ERROR  src.node.api_clients.base_client:base_client.py:16 HTTP error occurred: 404 Client Error: Not Found for url: http://127.0.0.1:3741/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1. Response content: b''
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 81850c7894f0
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 2683e90be550
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:73 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:74 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-03-20_04-41-18__27dc2e5c-e023-4d70-a3aa-7f2b251a692e__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-03-20_04-41-18__27dc2e5c-e023-4d70-a3aa-7f2b251a692e__wakuorg_nwaku:latest.log
DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_cant_publish_on_not_subscribed_shard with id: 2026-03-20_04-41-23__26e1e034-fa2d-4e6a-a26d-e980b941f2cd
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-20_04-41-23__26e1e034-fa2d-4e6a-a26d-e980b941f2cd__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.48.93
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['21058', '21059', '21060', '21061', '21062']
DEBUG  src.node.waku_node:waku_node.py:72 Cluster id was resolved to: 2
DEBUG  src.node.waku_node:waku_node.py:163 Removing pubsub-topic from nwaku args
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 21058:21058 -p 21059:21059 -p 21060:21060 -p 21061:21061 -p 21062:21062 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=21060 --rest-port=21058 --tcp-port=21059 --discv5-udp-port=21061 --rest-address=0.0.0.0 --nat=extip:172.18.48.93 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=ca3298eae5635af2063fdaffd09efa7d9f1a78eee51ba7daa5d869ecf5f71c0d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=21062 --metrics-logging=true --relay=true --filter=true
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.48.93 waku 9e8aefebcebc92f7dd8156100a8c283fb03880b7e3d17a40f150539a12539030
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 81850c7894f0. Exiting log stream.
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 9e8aefebcebc. Setting up logs at ./log/docker/node1_2026-03-20_04-41-23__26e1e034-fa2d-4e6a-a26d-e980b941f2cd__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 21058
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 2683e90be550. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21058/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21058/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21058/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.48.93/tcp/21059/p2p/16Uiu2HAmT9t46PpTiNt38okJt1PrdiubzuJQ5FyXNid3WCtnd95J","/ip4/172.18.48.93/tcp/21060/ws/p2p/16Uiu2HAmT9t46PpTiNt38okJt1PrdiubzuJQ5FyXNid3WCtnd95J"],"enrUri":"enr:-L24QDN0ueIcy6XHvlmsmZmNqtDH6ngJM9I4wuAzn1mCHCrXBZWB36MGndc9Q51uGrkUz5_6RFDjko1w2GX07os1dsgCgmlkgnY0gmlwhKwSMF2KbXVsdGlhZGRyc5YACASsEjBdBlJDAAoErBIwXQZSRN0DgnJzhQACAQAAiXNlY3AyNTZrMaED12pXSiM1dS6QqG9MYmxexrO77vWD4Ue5xzpLDitr_amDdGNwglJDg3VkcIJSRYV3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node2_2026-03-20_04-41-23__26e1e034-fa2d-4e6a-a26d-e980b941f2cd__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.223.232
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['40459', '40460', '40461', '40462', '40463']
DEBUG  src.node.waku_node:waku_node.py:72 Cluster id was resolved to: 2
DEBUG  src.node.waku_node:waku_node.py:163 Removing pubsub-topic from nwaku args
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 40459:40459 -p 40460:40460 -p 40461:40461 -p 40462:40462 -p 40463:40463 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=40461 --rest-port=40459 --tcp-port=40460 --discv5-udp-port=40462 --rest-address=0.0.0.0 --nat=extip:172.18.223.232 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=db2d70e4de57fbaf0d18f7830aec1a6b5f9cdf0d2679d0b47ec5bd1dbaffcec4 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=40463 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QDN0ueIcy6XHvlmsmZmNqtDH6ngJM9I4wuAzn1mCHCrXBZWB36MGndc9Q51uGrkUz5_6RFDjko1w2GX07os1dsgCgmlkgnY0gmlwhKwSMF2KbXVsdGlhZGRyc5YACASsEjBdBlJDAAoErBIwXQZSRN0DgnJzhQACAQAAiXNlY3AyNTZrMaED12pXSiM1dS6QqG9MYmxexrO77vWD4Ue5xzpLDitr_amDdGNwglJDg3VkcIJSRYV3YWt1MgU
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.223.232 waku c1b3428569c34b14c686d8e4f6dc9a84542aaed427c81c5537ff047752388af2
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID c1b3428569c3. Setting up logs at ./log/docker/node2_2026-03-20_04-41-23__26e1e034-fa2d-4e6a-a26d-e980b941f2cd__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 40459
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:40459/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:40459/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:40459/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:40459/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:40459/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.223.232/tcp/40460/p2p/16Uiu2HAmVUFDnP7s3qw3P3C8h2DGN3wUmqdHbGqZBSVb1Y5tvh7y","/ip4/172.18.223.232/tcp/40461/ws/p2p/16Uiu2HAmVUFDnP7s3qw3P3C8h2DGN3wUmqdHbGqZBSVb1Y5tvh7y"],"enrUri":"enr:-L24QEpiG2ww3Apf5yOgSgFiF_jJ_8qC0b1DpPenXcaFRewyfcVjR7zc22nrH1TGsxLTDAvC2xWaOXjmoAPM-81BG3ECgmlkgnY0gmlwhKwS3-iKbXVsdGlhZGRyc5YACASsEt_oBp4MAAoErBLf6AaeDd0DgnJzhQACAQAAiXNlY3AyNTZrMaED-dYaYjxSwiabr6NQOmxhRt2Vdkgpq0pqZ5Hx4XuZ2eyDdGNwgp4Mg3VkcIKeDoV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40459/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.48.93/tcp/21059/p2p/16Uiu2HAmT9t46PpTiNt38okJt1PrdiubzuJQ5FyXNid3WCtnd95J"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:21058/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40459/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:21058/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.libs.common:common.py:47 Sleeping for 0.1 seconds
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21058/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
ERROR  src.node.api_clients.base_client:base_client.py:16 HTTP error occurred: 404 Client Error: Not Found for url: http://127.0.0.1:21058/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1. Response content: b''
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 9e8aefebcebc
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id c1b3428569c3
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:73 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:74 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-03-20_04-41-23__26e1e034-fa2d-4e6a-a26d-e980b941f2cd__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-03-20_04-41-23__26e1e034-fa2d-4e6a-a26d-e980b941f2cd__wakuorg_nwaku:latest.log
DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_cant_publish_on_not_subscribed_shard with id: 2026-03-20_04-41-28__fe8de347-f760-4b42-85a8-994b0e9a15f1
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-20_04-41-28__fe8de347-f760-4b42-85a8-994b0e9a15f1__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.193.231
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['44092', '44093', '44094', '44095', '44096']
DEBUG  src.node.waku_node:waku_node.py:72 Cluster id was resolved to: 2
DEBUG  src.node.waku_node:waku_node.py:163 Removing pubsub-topic from nwaku args
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 44092:44092 -p 44093:44093 -p 44094:44094 -p 44095:44095 -p 44096:44096 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=44094 --rest-port=44092 --tcp-port=44093 --discv5-udp-port=44095 --rest-address=0.0.0.0 --nat=extip:172.18.193.231 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=926a434cdfcf8ba27f7602fb7ca6ebaef023d42b9e8a8cad38e49c24679bf5c1 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=44096 --metrics-logging=true --relay=true --filter=true
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.193.231 waku 4ccb34b2f9c7b2d63c927ed78f794c63d6f95c30871e2083ed5d728ef64671d6
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 4ccb34b2f9c7. Setting up logs at ./log/docker/node1_2026-03-20_04-41-28__fe8de347-f760-4b42-85a8-994b0e9a15f1__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 44092
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 9e8aefebcebc. Exiting log stream.
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container c1b3428569c3. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.193.231/tcp/44093/p2p/16Uiu2HAm8iAWsvJTKykAXh9g5FPV6xzjxxwgYcXRxwZAp7fmuVcW","/ip4/172.18.193.231/tcp/44094/ws/p2p/16Uiu2HAm8iAWsvJTKykAXh9g5FPV6xzjxxwgYcXRxwZAp7fmuVcW"],"enrUri":"enr:-L24QAv9VVfgetTe6wxX-S5zfL4wUi6H-kB2EDk8Rn-aQog_Kn_e6wuD4Z7YsYxO3s_WpGMmZZFqNIgBLuaKZY_9v3YCgmlkgnY0gmlwhKwSweeKbXVsdGlhZGRyc5YACASsEsHnBqw9AAoErBLB5wasPt0DgnJzhQACAQAAiXNlY3AyNTZrMaECxWF9GuMKjWr6vembQFKh4efW7wWL2BMadau4ZUjKyJuDdGNwgqw9g3VkcIKsP4V3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node2_2026-03-20_04-41-28__fe8de347-f760-4b42-85a8-994b0e9a15f1__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.124.100
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['32249', '32250', '32251', '32252', '32253']
DEBUG  src.node.waku_node:waku_node.py:72 Cluster id was resolved to: 2
DEBUG  src.node.waku_node:waku_node.py:163 Removing pubsub-topic from nwaku args
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 32249:32249 -p 32250:32250 -p 32251:32251 -p 32252:32252 -p 32253:32253 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=32251 --rest-port=32249 --tcp-port=32250 --discv5-udp-port=32252 --rest-address=0.0.0.0 --nat=extip:172.18.124.100 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=b1ccefd7870ed1ce5915009dc6bdbf4d4eb4e84ed7a3acbea194c7f439af149f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=32253 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QAv9VVfgetTe6wxX-S5zfL4wUi6H-kB2EDk8Rn-aQog_Kn_e6wuD4Z7YsYxO3s_WpGMmZZFqNIgBLuaKZY_9v3YCgmlkgnY0gmlwhKwSweeKbXVsdGlhZGRyc5YACASsEsHnBqw9AAoErBLB5wasPt0DgnJzhQACAQAAiXNlY3AyNTZrMaECxWF9GuMKjWr6vembQFKh4efW7wWL2BMadau4ZUjKyJuDdGNwgqw9g3VkcIKsP4V3YWt1MgU
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.124.100 waku 8500e1285d5de8c9d9b04a05d678cd4a8319226844aced88c4f3d74d9d7509cf
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 8500e1285d5d. Setting up logs at ./log/docker/node2_2026-03-20_04-41-28__fe8de347-f760-4b42-85a8-994b0e9a15f1__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 32249
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:32249/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:32249/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:32249/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:32249/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:32249/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.124.100/tcp/32250/p2p/16Uiu2HAm55wAj4DVStGbk17v9vFYpBtutzngMfFVSsivBEhvUZP6","/ip4/172.18.124.100/tcp/32251/ws/p2p/16Uiu2HAm55wAj4DVStGbk17v9vFYpBtutzngMfFVSsivBEhvUZP6"],"enrUri":"enr:-L24QKBJPGJKn847rcPFtYQoj2qJrMBulCxjsBLYH0rElNwPF6rPwlXZZ0EHOiyY_MQaPBkAPOTiegR4vm7lwqdXasACgmlkgnY0gmlwhKwSfGSKbXVsdGlhZGRyc5YACASsEnxkBn36AAoErBJ8ZAZ9-90DgnJzhQACAQAAiXNlY3AyNTZrMaECj4ZtQERzIebcP-OFABgUjRhU0Apa1IdNWAyXtbf9qkmDdGNwgn36g3VkcIJ9_IV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:32249/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.193.231/tcp/44093/p2p/16Uiu2HAm8iAWsvJTKykAXh9g5FPV6xzjxxwgYcXRxwZAp7fmuVcW"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:44092/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:32249/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:44092/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.libs.common:common.py:47 Sleeping for 0.1 seconds
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:44092/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
ERROR  src.node.api_clients.base_client:base_client.py:16 HTTP error occurred: 404 Client Error: Not Found for url: http://127.0.0.1:44092/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1. Response content: b''
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 4ccb34b2f9c7
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 8500e1285d5d
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:73 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:74 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-03-20_04-41-28__fe8de347-f760-4b42-85a8-994b0e9a15f1__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-03-20_04-41-28__fe8de347-f760-4b42-85a8-994b0e9a15f1__wakuorg_nwaku:latest.log