315 lines
55 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_2_nodes_different_content_topic_same_shard with id: 2026-03-20_04-35-56__cbc11bcf-93c3-4fae-89d1-c806766caaf6
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-20_04-35-56__cbc11bcf-93c3-4fae-89d1-c806766caaf6__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.144.112
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['38419', '38420', '38421', '38422', '38423']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 38419:38419 -p 38420:38420 -p 38421:38421 -p 38422:38422 -p 38423:38423 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=38421 --rest-port=38419 --tcp-port=38420 --discv5-udp-port=38422 --rest-address=0.0.0.0 --nat=extip:172.18.144.112 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=aaa05ea4efdfc5bfbccaef5cb54b44fcff1fed0a3ccafebd16225b9ef83863f1 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=38423 --metrics-logging=true --relay=true --filter=true --content-topic=/newsService/1.0/weekly/protobuf
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 2b03d72cf5aa. Exiting log stream.
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.144.112 waku d9b79cb87949a9f54114df864430958a3ea8645017d887c48d1fed63038b563d
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID d9b79cb87949. Setting up logs at ./log/docker/node1_2026-03-20_04-35-56__cbc11bcf-93c3-4fae-89d1-c806766caaf6__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 38419
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 1d84881dc5ed. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.144.112/tcp/38420/p2p/16Uiu2HAm64MnZTkLrGET56JNEWiixV8r37o4jQXFN8tYDASd2tTy","/ip4/172.18.144.112/tcp/38421/ws/p2p/16Uiu2HAm64MnZTkLrGET56JNEWiixV8r37o4jQXFN8tYDASd2tTy"],"enrUri":"enr:-L24QNfWi8hcJ_8TSrnpB8JUFPdfoSMXnIwqewwepft5rpM6LpJAtsnE6tXVCNnSDb15vHPSeEo7IYKC2BvrADLCPIICgmlkgnY0gmlwhKwSkHCKbXVsdGlhZGRyc5YACASsEpBwBpYUAAoErBKQcAaWFd0DgnJzhQACAQAAiXNlY3AyNTZrMaECnfrJnLvsneehODWRnjfmm3omy6KdUx7SybFxU_HieJCDdGNwgpYUg3VkcIKWFoV3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node2_2026-03-20_04-35-56__cbc11bcf-93c3-4fae-89d1-c806766caaf6__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.152.235
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['65271', '65272', '65273', '65274', '65275']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 65271:65271 -p 65272:65272 -p 65273:65273 -p 65274:65274 -p 65275:65275 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=65273 --rest-port=65271 --tcp-port=65272 --discv5-udp-port=65274 --rest-address=0.0.0.0 --nat=extip:172.18.152.235 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=d25ed6aefd03a710f7cddf544ccfa6c87b90b451049949e8bb86e83ce80b88ce --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=65275 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNfWi8hcJ_8TSrnpB8JUFPdfoSMXnIwqewwepft5rpM6LpJAtsnE6tXVCNnSDb15vHPSeEo7IYKC2BvrADLCPIICgmlkgnY0gmlwhKwSkHCKbXVsdGlhZGRyc5YACASsEpBwBpYUAAoErBKQcAaWFd0DgnJzhQACAQAAiXNlY3AyNTZrMaECnfrJnLvsneehODWRnjfmm3omy6KdUx7SybFxU_HieJCDdGNwgpYUg3VkcIKWFoV3YWt1MgU --content-topic=/newsService/1.0/alerts/xml
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.152.235 waku 186f04002417d8465f3190026bcdfebe697476f64103c1fd05f11c360abe0f97
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 186f04002417. Setting up logs at ./log/docker/node2_2026-03-20_04-35-56__cbc11bcf-93c3-4fae-89d1-c806766caaf6__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 65271
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:65271/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:65271/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.152.235/tcp/65272/p2p/16Uiu2HAm4bz79MNG2mJgvLugnAhioejBunTQv8F13VQtyrViQywz","/ip4/172.18.152.235/tcp/65273/ws/p2p/16Uiu2HAm4bz79MNG2mJgvLugnAhioejBunTQv8F13VQtyrViQywz"],"enrUri":"enr:-L24QCI_SM6FsTXwpsu2AtYYxpcnkpsmzb7gWx-mEUcgCmUddPEX14UzYmB6Ec-9nIRU3O3WSZ4FnUSlyrnBsrA38voCgmlkgnY0gmlwhKwSmOuKbXVsdGlhZGRyc5YACASsEpjrBv74AAoErBKY6wb--d0DgnJzhQACAQAAiXNlY3AyNTZrMaECiF18hQ6iRNNX1qov_VUAn5zAodJ8VWDKLRL8VO6u4j2DdGNwgv74g3VkcIL--oV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:65271/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.144.112/tcp/38420/p2p/16Uiu2HAm64MnZTkLrGET56JNEWiixV8r37o4jQXFN8tYDASd2tTy"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:38419/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/newsService/1.0/weekly/protobuf"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:65271/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/newsService/1.0/alerts/xml"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:38419/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/newsService/1.0/weekly/protobuf", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.libs.common:common.py:47 Sleeping for 0.1 seconds
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38419/relay/v1/auto/messages/%2FnewsService%2F1.0%2Fweekly%2Fprotobuf" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/newsService/1.0/weekly/protobuf","version":0,"timestamp":1773981363421363371,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:65271/relay/v1/auto/messages/%2FnewsService%2F1.0%2Fweekly%2Fprotobuf" -H "Content-Type: application/json" -d 'None'
ERROR  src.node.api_clients.base_client:base_client.py:16 HTTP error occurred: 404 Client Error: Not Found for url: http://127.0.0.1:65271/relay/v1/auto/messages/%2FnewsService%2F1.0%2Fweekly%2Fprotobuf. Response content: b'/newsService/1.0/weekly/protobuf'
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id d9b79cb87949
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 186f04002417
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:73 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:74 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-03-20_04-35-56__cbc11bcf-93c3-4fae-89d1-c806766caaf6__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-03-20_04-35-56__cbc11bcf-93c3-4fae-89d1-c806766caaf6__wakuorg_nwaku:latest.log
DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_2_nodes_different_content_topic_same_shard with id: 2026-03-20_04-36-04__c59b8e71-3039-453c-a2d8-d23a1c6c30f1
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-20_04-36-04__c59b8e71-3039-453c-a2d8-d23a1c6c30f1__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.149.219
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['21953', '21954', '21955', '21956', '21957']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 21953:21953 -p 21954:21954 -p 21955:21955 -p 21956:21956 -p 21957:21957 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=21955 --rest-port=21953 --tcp-port=21954 --discv5-udp-port=21956 --rest-address=0.0.0.0 --nat=extip:172.18.149.219 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=8ccbdc8a6e5a56e918e9bd8e8b8dc1b7ee6235abd25b7fa333a542fa9ea266b2 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=21957 --metrics-logging=true --relay=true --filter=true --content-topic=/newsService/1.0/weekly/protobuf
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container d9b79cb87949. Exiting log stream.
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.149.219 waku f5261ba9d442f6ec2538be8d4c70121332ca34611f914ba7a32a700fed15fb8f
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID f5261ba9d442. Setting up logs at ./log/docker/node1_2026-03-20_04-36-04__c59b8e71-3039-453c-a2d8-d23a1c6c30f1__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 21953
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 186f04002417. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21953/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21953/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21953/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.149.219/tcp/21954/p2p/16Uiu2HAkwRs2vUP9ZU2KihSiJnBKpjrMjp2oCL63SkNAdezHFW9i","/ip4/172.18.149.219/tcp/21955/ws/p2p/16Uiu2HAkwRs2vUP9ZU2KihSiJnBKpjrMjp2oCL63SkNAdezHFW9i"],"enrUri":"enr:-L24QKW9io0I3yYMA0Nd5CeQ4zWcb49sUgZZ-3r3UgbzWQKBKOiiWCrrNulgTEqsaEiyMqyOrX4febyMSzSHUKb8ivMCgmlkgnY0gmlwhKwSlduKbXVsdGlhZGRyc5YACASsEpXbBlXCAAoErBKV2wZVw90DgnJzhQACAQAAiXNlY3AyNTZrMaECHcPfYnooxHrlXx6EJO2Vcja0zrXivg2GRQtjE5tmDx2DdGNwglXCg3VkcIJVxIV3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node2_2026-03-20_04-36-04__c59b8e71-3039-453c-a2d8-d23a1c6c30f1__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.228.191
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['37612', '37613', '37614', '37615', '37616']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 37612:37612 -p 37613:37613 -p 37614:37614 -p 37615:37615 -p 37616:37616 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=37614 --rest-port=37612 --tcp-port=37613 --discv5-udp-port=37615 --rest-address=0.0.0.0 --nat=extip:172.18.228.191 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=f5fe2bab636ca51a0c566bc6b49befdd7a0c69937def9efa4ecf2984b8fbc4b4 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=37616 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QKW9io0I3yYMA0Nd5CeQ4zWcb49sUgZZ-3r3UgbzWQKBKOiiWCrrNulgTEqsaEiyMqyOrX4febyMSzSHUKb8ivMCgmlkgnY0gmlwhKwSlduKbXVsdGlhZGRyc5YACASsEpXbBlXCAAoErBKV2wZVw90DgnJzhQACAQAAiXNlY3AyNTZrMaECHcPfYnooxHrlXx6EJO2Vcja0zrXivg2GRQtjE5tmDx2DdGNwglXCg3VkcIJVxIV3YWt1MgU --content-topic=/newsService/1.0/alerts/xml
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.228.191 waku 87281a2af4debc5caf911fbfcbefccfe385386433daa7ca363cbedede3751f69
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 87281a2af4de. Setting up logs at ./log/docker/node2_2026-03-20_04-36-04__c59b8e71-3039-453c-a2d8-d23a1c6c30f1__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 37612
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:37612/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:37612/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.228.191/tcp/37613/p2p/16Uiu2HAmTDhyNVVkPGWgistc5dRnLDoSxXiZ6KynXRvH3hcPD7p3","/ip4/172.18.228.191/tcp/37614/ws/p2p/16Uiu2HAmTDhyNVVkPGWgistc5dRnLDoSxXiZ6KynXRvH3hcPD7p3"],"enrUri":"enr:-L24QI4ZhypVzyhu1RuxEQvntranYVE8FTNY14x1VmCC1rlTNCKj6iBYnIOusS3U6n4GQZSNWuis7wlOML_fvys5mKsCgmlkgnY0gmlwhKwS5L-KbXVsdGlhZGRyc5YACASsEuS_BpLtAAoErBLkvwaS7t0DgnJzhQACAQAAiXNlY3AyNTZrMaED2GVD3lqoy-zg49pobesoai778sa3x2xCie2iE2UvyKCDdGNwgpLtg3VkcIKS74V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:37612/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.149.219/tcp/21954/p2p/16Uiu2HAkwRs2vUP9ZU2KihSiJnBKpjrMjp2oCL63SkNAdezHFW9i"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:21953/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/newsService/1.0/weekly/protobuf"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:37612/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/newsService/1.0/alerts/xml"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:21953/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/newsService/1.0/weekly/protobuf", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.libs.common:common.py:47 Sleeping for 0.1 seconds
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21953/relay/v1/auto/messages/%2FnewsService%2F1.0%2Fweekly%2Fprotobuf" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/newsService/1.0/weekly/protobuf","version":0,"timestamp":1773981367515506309,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:37612/relay/v1/auto/messages/%2FnewsService%2F1.0%2Fweekly%2Fprotobuf" -H "Content-Type: application/json" -d 'None'
ERROR  src.node.api_clients.base_client:base_client.py:16 HTTP error occurred: 404 Client Error: Not Found for url: http://127.0.0.1:37612/relay/v1/auto/messages/%2FnewsService%2F1.0%2Fweekly%2Fprotobuf. Response content: b'/newsService/1.0/weekly/protobuf'
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id f5261ba9d442
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 87281a2af4de
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:73 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:74 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-03-20_04-36-04__c59b8e71-3039-453c-a2d8-d23a1c6c30f1__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-03-20_04-36-04__c59b8e71-3039-453c-a2d8-d23a1c6c30f1__wakuorg_nwaku:latest.log
DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_2_nodes_different_content_topic_same_shard with id: 2026-03-20_04-36-08__cf5e2a45-498e-44f7-8768-322d3045067b
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-20_04-36-08__cf5e2a45-498e-44f7-8768-322d3045067b__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.71.161
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['65073', '65074', '65075', '65076', '65077']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 65073:65073 -p 65074:65074 -p 65075:65075 -p 65076:65076 -p 65077:65077 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=65075 --rest-port=65073 --tcp-port=65074 --discv5-udp-port=65076 --rest-address=0.0.0.0 --nat=extip:172.18.71.161 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=f6c43daacacaa542e7fdb8d5d80994507f78a853f06c5f11cbadfde8cdb7effc --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=65077 --metrics-logging=true --relay=true --filter=true --content-topic=/newsService/1.0/weekly/protobuf
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container f5261ba9d442. Exiting log stream.
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.71.161 waku c736826bd7b83484741747cc7663bb349359d97925654a62db412def1c00e6e0
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID c736826bd7b8. Setting up logs at ./log/docker/node1_2026-03-20_04-36-08__cf5e2a45-498e-44f7-8768-322d3045067b__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 65073
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 87281a2af4de. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:65073/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:65073/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:65073/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:65073/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.71.161/tcp/65074/p2p/16Uiu2HAmK2XGEhXpm6fuWyoDXxEwHRdMaUXzDAsSqeZ8tYmb7vky","/ip4/172.18.71.161/tcp/65075/ws/p2p/16Uiu2HAmK2XGEhXpm6fuWyoDXxEwHRdMaUXzDAsSqeZ8tYmb7vky"],"enrUri":"enr:-L24QNSW4u1VbIZq4nrLXuHaV-FL3tfBVrp2X-Ke_pqlQCaRAB2SczvMKr1hzNzeKKf_b0i9ti0ZWLqq2Zq2yhNxipICgmlkgnY0gmlwhKwSR6GKbXVsdGlhZGRyc5YACASsEkehBv4yAAoErBJHoQb-M90DgnJzhQACAQAAiXNlY3AyNTZrMaEDXqpFmE31mt9JhHVxIIC5e18J6ewnRLt96_GPak0eVzqDdGNwgv4yg3VkcIL-NIV3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node2_2026-03-20_04-36-08__cf5e2a45-498e-44f7-8768-322d3045067b__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.182.2
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['20336', '20337', '20338', '20339', '20340']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 20336:20336 -p 20337:20337 -p 20338:20338 -p 20339:20339 -p 20340:20340 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=20338 --rest-port=20336 --tcp-port=20337 --discv5-udp-port=20339 --rest-address=0.0.0.0 --nat=extip:172.18.182.2 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=3e2c9a3ca6addefff124a3d8f3aaeec9eb2cef2a2d7a1dda7db1cb0b6bcc49dd --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=20340 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNSW4u1VbIZq4nrLXuHaV-FL3tfBVrp2X-Ke_pqlQCaRAB2SczvMKr1hzNzeKKf_b0i9ti0ZWLqq2Zq2yhNxipICgmlkgnY0gmlwhKwSR6GKbXVsdGlhZGRyc5YACASsEkehBv4yAAoErBJHoQb-M90DgnJzhQACAQAAiXNlY3AyNTZrMaEDXqpFmE31mt9JhHVxIIC5e18J6ewnRLt96_GPak0eVzqDdGNwgv4yg3VkcIL-NIV3YWt1MgU --content-topic=/newsService/1.0/alerts/xml
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.182.2 waku acf47dd18b79be4c8cd4cb8be5fb3f726452d49afe8dc835c1c80c2790e688a7
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID acf47dd18b79. Setting up logs at ./log/docker/node2_2026-03-20_04-36-08__cf5e2a45-498e-44f7-8768-322d3045067b__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 20336
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:20336/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:20336/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.182.2/tcp/20337/p2p/16Uiu2HAmPCEpT6GcCx1WPTd5ei1RBeq6SVrV77Carfi8Sa4USFYQ","/ip4/172.18.182.2/tcp/20338/ws/p2p/16Uiu2HAmPCEpT6GcCx1WPTd5ei1RBeq6SVrV77Carfi8Sa4USFYQ"],"enrUri":"enr:-L24QCywqeNvKXCTKuj51CM4pkV25yQ4QYbXg8uP_KJdqKsQMvF99XFyjNFDl6vnO6d9c_f1ojJdaTCxwIzb--VhBYsCgmlkgnY0gmlwhKwStgKKbXVsdGlhZGRyc5YACASsErYCBk9xAAoErBK2AgZPct0DgnJzhQACAQAAiXNlY3AyNTZrMaEDnJY70zlsIljDEgGKe0-cAGWfppFKtY2ya0MUdQdXBE2DdGNwgk9xg3VkcIJPc4V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:20336/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.71.161/tcp/65074/p2p/16Uiu2HAmK2XGEhXpm6fuWyoDXxEwHRdMaUXzDAsSqeZ8tYmb7vky"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:65073/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/newsService/1.0/weekly/protobuf"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:20336/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/newsService/1.0/alerts/xml"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:65073/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/newsService/1.0/weekly/protobuf", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.libs.common:common.py:47 Sleeping for 0.1 seconds
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:65073/relay/v1/auto/messages/%2FnewsService%2F1.0%2Fweekly%2Fprotobuf" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/newsService/1.0/weekly/protobuf","version":0,"timestamp":1773981371800865784,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:20336/relay/v1/auto/messages/%2FnewsService%2F1.0%2Fweekly%2Fprotobuf" -H "Content-Type: application/json" -d 'None'
ERROR  src.node.api_clients.base_client:base_client.py:16 HTTP error occurred: 404 Client Error: Not Found for url: http://127.0.0.1:20336/relay/v1/auto/messages/%2FnewsService%2F1.0%2Fweekly%2Fprotobuf. Response content: b'/newsService/1.0/weekly/protobuf'
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id c736826bd7b8
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id acf47dd18b79
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:73 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:74 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-03-20_04-36-08__cf5e2a45-498e-44f7-8768-322d3045067b__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-03-20_04-36-08__cf5e2a45-498e-44f7-8768-322d3045067b__wakuorg_nwaku:latest.log