179 lines
29 KiB
Plaintext

2026-03-11 04:41:51.197 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-11 04:41:51.198 DEBUG [tests.conftest] Running test: test_cant_publish_on_not_subscribed_shard with id: 2026-03-11_04-41-51__aa3709fd-0576-443c-8b1a-97578582f538
2026-03-11 04:41:51.200 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-11 04:41:51.200 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-11 04:41:51.200 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-11 04:41:51.202 ERROR [src.node.docker_mananger] Max retries reached for container f5a96b7fca70. Exiting log stream.
2026-03-11 04:41:51.211 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:41:51.212 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-11_04-41-51__aa3709fd-0576-443c-8b1a-97578582f538__wakuorg_nwaku:latest.log
2026-03-11 04:41:51.213 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:41:51.213 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:41:51.216 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:41:51.217 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.56.117
2026-03-11 04:41:51.218 DEBUG [src.node.docker_mananger] Generated ports ['18651', '18652', '18653', '18654', '18655']
2026-03-11 04:41:51.219 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-11 04:41:51.219 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-11 04:41:51.220 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:41:51.220 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:41:51.221 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:41:51.222 DEBUG [src.node.docker_mananger] docker run -i -t -p 18651:18651 -p 18652:18652 -p 18653:18653 -p 18654:18654 -p 18655:18655 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=18653 --rest-port=18651 --tcp-port=18652 --discv5-udp-port=18654 --rest-address=0.0.0.0 --nat=extip:172.18.56.117 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=65a9e6feaecdedd26349ce6d43983fef1d56e4ca54294ebbc26efc7ce3af96ce --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=18655 --metrics-logging=true --relay=true --filter=true
2026-03-11 04:41:51.437 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.56.117 waku 04a8e49a23a5dae5beb9f865213ed3c6c7fbb890964e255d729e2f229924ee32
2026-03-11 04:41:51.475 DEBUG [src.node.docker_mananger] Container started with ID 04a8e49a23a5. Setting up logs at ./log/docker/node1_2026-03-11_04-41-51__aa3709fd-0576-443c-8b1a-97578582f538__wakuorg_nwaku:latest.log
2026-03-11 04:41:51.476 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 18651
2026-03-11 04:41:51.476 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:41:52.113 ERROR [src.node.docker_mananger] Max retries reached for container 54ce8d2c7625. Exiting log stream.
2026-03-11 04:41:52.476 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:18651/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:52.479 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:41:52.579 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:18651/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:52.583 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:41:52.583 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:41:52.585 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:18651/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:52.590 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.56.117/tcp/18652/p2p/16Uiu2HAm7E6jhc2rBmWvDAcRG87xzkSVY1NjPfBVJHAGajRyzzxR","/ip4/172.18.56.117/tcp/18653/ws/p2p/16Uiu2HAm7E6jhc2rBmWvDAcRG87xzkSVY1NjPfBVJHAGajRyzzxR"],"enrUri":"enr:-L24QL04NnS4fmMKwXPNC3aZ9KG-R_Q0DmOklq13CF2xMfY_fI8FDEAZSOgU1lkhbtvdyFbpU90ERwwyNYQZz4Au4swCgmlkgnY0gmlwhKwSOHWKbXVsdGlhZGRyc5YACASsEjh1BkjcAAoErBI4dQZI3d0DgnJzhQACAQAAiXNlY3AyNTZrMaECr1VEKdJNs5Mgs44q-QuccLWxjQh5c212LoSBR5Lp8PqDdGNwgkjcg3VkcIJI3oV3YWt1MgU"}'
2026-03-11 04:41:52.591 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:41:52.599 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:41:52.599 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-11_04-41-51__aa3709fd-0576-443c-8b1a-97578582f538__wakuorg_nwaku:latest.log
2026-03-11 04:41:52.599 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:41:52.600 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:41:52.601 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:41:52.601 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.106.197
2026-03-11 04:41:52.601 DEBUG [src.node.docker_mananger] Generated ports ['8996', '8997', '8998', '8999', '9000']
2026-03-11 04:41:52.601 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-11 04:41:52.602 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-11 04:41:52.602 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:41:52.602 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:41:52.602 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:41:52.602 DEBUG [src.node.docker_mananger] docker run -i -t -p 8996:8996 -p 8997:8997 -p 8998:8998 -p 8999:8999 -p 9000:9000 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=8998 --rest-port=8996 --tcp-port=8997 --discv5-udp-port=8999 --rest-address=0.0.0.0 --nat=extip:172.18.106.197 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=6b1a494267bc3c6f17fdc997c953b85b2fecde206db4ca40362c6fb77a5ea85b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=9000 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QL04NnS4fmMKwXPNC3aZ9KG-R_Q0DmOklq13CF2xMfY_fI8FDEAZSOgU1lkhbtvdyFbpU90ERwwyNYQZz4Au4swCgmlkgnY0gmlwhKwSOHWKbXVsdGlhZGRyc5YACASsEjh1BkjcAAoErBI4dQZI3d0DgnJzhQACAQAAiXNlY3AyNTZrMaECr1VEKdJNs5Mgs44q-QuccLWxjQh5c212LoSBR5Lp8PqDdGNwgkjcg3VkcIJI3oV3YWt1MgU
2026-03-11 04:41:52.817 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.106.197 waku 96eb2513bf523b9ba8d6ca813ea38f0f13a12ba76c93e83e5839b11d0bf1cabd
2026-03-11 04:41:52.852 DEBUG [src.node.docker_mananger] Container started with ID 96eb2513bf52. Setting up logs at ./log/docker/node2_2026-03-11_04-41-51__aa3709fd-0576-443c-8b1a-97578582f538__wakuorg_nwaku:latest.log
2026-03-11 04:41:52.852 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 8996
2026-03-11 04:41:52.853 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:41:53.853 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8996/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:53.895 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:41:53.996 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8996/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:53.998 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:41:53.998 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:41:53.999 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8996/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:54.001 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.106.197/tcp/8997/p2p/16Uiu2HAmRtMkbToTQvv9dqwCe3X2VbJCWwVN8BSNt2WcyGY3yoh3","/ip4/172.18.106.197/tcp/8998/ws/p2p/16Uiu2HAmRtMkbToTQvv9dqwCe3X2VbJCWwVN8BSNt2WcyGY3yoh3"],"enrUri":"enr:-L24QDgjUPVnUv6lLseWUmGX_YDetO6ErdWXIdmkFB34yFGpNSL0HYYt1hxkTRF4v0NlqgpMoS9dqO6pu7vocwYw994CgmlkgnY0gmlwhKwSasWKbXVsdGlhZGRyc5YACASsEmrFBiMlAAoErBJqxQYjJt0DgnJzhQACAQAAiXNlY3AyNTZrMaEDxJSt_qYyYEDFEbwynIEf46lPvtOZ-4StLqSuZuSz4YqDdGNwgiMlg3VkcIIjJ4V3YWt1MgE"}'
2026-03-11 04:41:54.001 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:41:54.001 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:8996/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.56.117/tcp/18652/p2p/16Uiu2HAm7E6jhc2rBmWvDAcRG87xzkSVY1NjPfBVJHAGajRyzzxR"]'
2026-03-11 04:41:54.025 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:41:54.026 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:18651/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-11 04:41:54.045 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:41:54.046 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:8996/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-11 04:41:54.051 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:41:54.053 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:18651/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:41:54.067 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:41:54.068 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-11 04:41:54.168 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-11 04:41:54.169 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:18651/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:54.171 ERROR [src.node.api_clients.base_client] HTTP error occurred: 404 Client Error: Not Found for url: http://127.0.0.1:18651/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1. Response content: b''
2026-03-11 04:41:54.246 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-11 04:41:54.247 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-11 04:41:54.247 DEBUG [src.node.waku_node] Stopping container with id 04a8e49a23a5
2026-03-11 04:41:54.856 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:41:54.856 DEBUG [src.node.waku_node] Stopping container with id 96eb2513bf52
2026-03-11 04:41:55.443 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:41:55.445 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-11 04:41:55.477 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:41:55.503 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:41:55.505 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-11 04:41:55.507 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-11 04:41:55.507 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-11_04-41-51__aa3709fd-0576-443c-8b1a-97578582f538__wakuorg_nwaku:latest.log
2026-03-11 04:41:55.509 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-11_04-41-51__aa3709fd-0576-443c-8b1a-97578582f538__wakuorg_nwaku:latest.log
2026-03-11 04:41:55.512 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-11 04:41:55.513 DEBUG [tests.conftest] Running test: test_cant_publish_on_not_subscribed_shard with id: 2026-03-11_04-41-55__f830e57b-f555-4555-aa09-1478b4f9d491
2026-03-11 04:41:55.513 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-11 04:41:55.513 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-11 04:41:55.514 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-11 04:41:55.521 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:41:55.521 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-11_04-41-55__f830e57b-f555-4555-aa09-1478b4f9d491__wakuorg_nwaku:latest.log
2026-03-11 04:41:55.522 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:41:55.522 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:41:55.523 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:41:55.523 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.183.139
2026-03-11 04:41:55.523 DEBUG [src.node.docker_mananger] Generated ports ['23285', '23286', '23287', '23288', '23289']
2026-03-11 04:41:55.524 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-11 04:41:55.524 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-11 04:41:55.524 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:41:55.524 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:41:55.524 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:41:55.524 DEBUG [src.node.docker_mananger] docker run -i -t -p 23285:23285 -p 23286:23286 -p 23287:23287 -p 23288:23288 -p 23289:23289 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=23287 --rest-port=23285 --tcp-port=23286 --discv5-udp-port=23288 --rest-address=0.0.0.0 --nat=extip:172.18.183.139 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=df0d31cad2edd35c28f0f0f74e5f5c1cecbdfccbefbb3eeeeb89a579fbd28f0f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=23289 --metrics-logging=true --relay=true --filter=true
2026-03-11 04:41:55.710 ERROR [src.node.docker_mananger] Max retries reached for container 04a8e49a23a5. Exiting log stream.
2026-03-11 04:41:55.713 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.183.139 waku afe9010282f8ceaec5ce7ef3458397bf5eb87d65eabc756fdd08e134ffafe075
2026-03-11 04:41:55.753 DEBUG [src.node.docker_mananger] Container started with ID afe9010282f8. Setting up logs at ./log/docker/node1_2026-03-11_04-41-55__f830e57b-f555-4555-aa09-1478b4f9d491__wakuorg_nwaku:latest.log
2026-03-11 04:41:55.754 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 23285
2026-03-11 04:41:55.755 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:41:56.314 ERROR [src.node.docker_mananger] Max retries reached for container 96eb2513bf52. Exiting log stream.
2026-03-11 04:41:56.755 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23285/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:56.759 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:41:56.860 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23285/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:56.862 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:41:56.963 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23285/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:56.965 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:41:57.065 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23285/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:57.068 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:41:57.168 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23285/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:57.170 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:41:57.271 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23285/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:57.273 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:41:57.374 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23285/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:57.376 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:41:57.477 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23285/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:57.514 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:41:57.616 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23285/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:57.618 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:41:57.618 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:41:57.619 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23285/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:57.621 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.183.139/tcp/23286/p2p/16Uiu2HAmHzLgc86aAksEkQ2j3nTyFvDzaz2AhH42xpEL7E6PLie3","/ip4/172.18.183.139/tcp/23287/ws/p2p/16Uiu2HAmHzLgc86aAksEkQ2j3nTyFvDzaz2AhH42xpEL7E6PLie3"],"enrUri":"enr:-L24QO4HbcFbZQARiM18A4VYY74Vxg2X904HS-3hkXYuT05zNIubgpuTPyKfHE-_PUpJRls0fo2p42pDCvk7-0F_gfsCgmlkgnY0gmlwhKwSt4uKbXVsdGlhZGRyc5YACASsEreLBlr2AAoErBK3iwZa990DgnJzhQACAQAAiXNlY3AyNTZrMaEDTz917RyBT3bQVnHYgBjlHHdH7Ov6puHRFHsUAXktWOCDdGNwglr2g3VkcIJa-IV3YWt1MgU"}'
2026-03-11 04:41:57.621 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:41:57.629 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:41:57.629 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-11_04-41-55__f830e57b-f555-4555-aa09-1478b4f9d491__wakuorg_nwaku:latest.log
2026-03-11 04:41:57.629 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:41:57.629 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:41:57.630 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:41:57.631 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.125.91
2026-03-11 04:41:57.631 DEBUG [src.node.docker_mananger] Generated ports ['19432', '19433', '19434', '19435', '19436']
2026-03-11 04:41:57.631 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-11 04:41:57.631 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-11 04:41:57.631 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:41:57.631 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:41:57.631 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:41:57.632 DEBUG [src.node.docker_mananger] docker run -i -t -p 19432:19432 -p 19433:19433 -p 19434:19434 -p 19435:19435 -p 19436:19436 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=19434 --rest-port=19432 --tcp-port=19433 --discv5-udp-port=19435 --rest-address=0.0.0.0 --nat=extip:172.18.125.91 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=de30f7baf316e6b67c0bec851c7ed1cf2ecc42f52d2ac1ef5e65cdfce69cc62a --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=19436 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QO4HbcFbZQARiM18A4VYY74Vxg2X904HS-3hkXYuT05zNIubgpuTPyKfHE-_PUpJRls0fo2p42pDCvk7-0F_gfsCgmlkgnY0gmlwhKwSt4uKbXVsdGlhZGRyc5YACASsEreLBlr2AAoErBK3iwZa990DgnJzhQACAQAAiXNlY3AyNTZrMaEDTz917RyBT3bQVnHYgBjlHHdH7Ov6puHRFHsUAXktWOCDdGNwglr2g3VkcIJa-IV3YWt1MgU
2026-03-11 04:41:57.835 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.125.91 waku a152aaf49e30471df83e4ff04f41cfc6547a9dc8c98c6e87d98efe65b76a2b60
2026-03-11 04:41:57.869 DEBUG [src.node.docker_mananger] Container started with ID a152aaf49e30. Setting up logs at ./log/docker/node2_2026-03-11_04-41-55__f830e57b-f555-4555-aa09-1478b4f9d491__wakuorg_nwaku:latest.log
2026-03-11 04:41:57.869 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 19432
2026-03-11 04:41:57.870 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:41:58.870 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19432/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:58.873 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:41:58.973 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19432/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:58.976 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:41:58.976 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:41:58.976 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19432/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:58.979 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.125.91/tcp/19433/p2p/16Uiu2HAm1UFESLXHE4C94z2Nunve6FR6n436DaejytXouqFGG5M3","/ip4/172.18.125.91/tcp/19434/ws/p2p/16Uiu2HAm1UFESLXHE4C94z2Nunve6FR6n436DaejytXouqFGG5M3"],"enrUri":"enr:-L24QM7SH-7jNJPplwr_DcDaua82IpJTCYPjAnan-AMP9Xu_EKWNUEJtw2Gd4shIqjl-zV2CtQLCrbRcWA8zPgEmkUECgmlkgnY0gmlwhKwSfVuKbXVsdGlhZGRyc5YACASsEn1bBkvpAAoErBJ9WwZL6t0DgnJzhQACAQAAiXNlY3AyNTZrMaECWc7hwvUBlk3ddog8TZJnOBcZ16NMmS40BpFO5SGpaCKDdGNwgkvpg3VkcIJL64V3YWt1MgE"}'
2026-03-11 04:41:58.979 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:41:58.979 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19432/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.183.139/tcp/23286/p2p/16Uiu2HAmHzLgc86aAksEkQ2j3nTyFvDzaz2AhH42xpEL7E6PLie3"]'
2026-03-11 04:41:59.006 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:41:59.007 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23285/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-11 04:41:59.016 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:41:59.017 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19432/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-11 04:41:59.024 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:41:59.027 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23285/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:41:59.036 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:41:59.037 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-11 04:41:59.139 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-11 04:41:59.140 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23285/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:41:59.142 ERROR [src.node.api_clients.base_client] HTTP error occurred: 404 Client Error: Not Found for url: http://127.0.0.1:23285/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1. Response content: b''
2026-03-11 04:41:59.181 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-11 04:41:59.182 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-11 04:41:59.182 DEBUG [src.node.waku_node] Stopping container with id afe9010282f8
2026-03-11 04:41:59.779 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:41:59.780 DEBUG [src.node.waku_node] Stopping container with id a152aaf49e30
2026-03-11 04:42:00.352 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:42:00.354 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-11 04:42:00.387 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:42:00.425 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:42:00.427 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-11 04:42:00.428 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-11 04:42:00.432 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-11_04-41-55__f830e57b-f555-4555-aa09-1478b4f9d491__wakuorg_nwaku:latest.log
2026-03-11 04:42:00.435 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-11_04-41-55__f830e57b-f555-4555-aa09-1478b4f9d491__wakuorg_nwaku:latest.log