169 lines
27 KiB
Plaintext

2026-03-09 04:47:04.771 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-09 04:47:04.771 DEBUG [tests.conftest] Running test: test_cant_publish_on_not_subscribed_shard with id: 2026-03-09_04-47-04__3c62147d-f24e-48c1-88a1-c8e8207a7900
2026-03-09 04:47:04.771 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-09 04:47:04.772 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-09 04:47:04.772 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-09 04:47:04.779 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-09 04:47:04.780 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-09_04-47-04__3c62147d-f24e-48c1-88a1-c8e8207a7900__wakuorg_nwaku:latest.log
2026-03-09 04:47:04.780 DEBUG [src.node.waku_node] Starting Node...
2026-03-09 04:47:04.781 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-09 04:47:04.783 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-09 04:47:04.783 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.126.7
2026-03-09 04:47:04.783 DEBUG [src.node.docker_mananger] Generated ports ['4551', '4552', '4553', '4554', '4555']
2026-03-09 04:47:04.784 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-09 04:47:04.784 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-09 04:47:04.784 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-09 04:47:04.784 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-09 04:47:04.784 DEBUG [src.node.waku_node] Using volumes []
2026-03-09 04:47:04.784 DEBUG [src.node.docker_mananger] docker run -i -t -p 4551:4551 -p 4552:4552 -p 4553:4553 -p 4554:4554 -p 4555:4555 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=4553 --rest-port=4551 --tcp-port=4552 --discv5-udp-port=4554 --rest-address=0.0.0.0 --nat=extip:172.18.126.7 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=373dd87a779e29378de22579ea67b5b4856ca4fbb31fa8bbae0d9e6d1f298620 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=4555 --metrics-logging=true --relay=true --filter=true
2026-03-09 04:47:04.899 ERROR [src.node.docker_mananger] Max retries reached for container 3820d9306c85. Exiting log stream.
2026-03-09 04:47:04.982 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.126.7 waku 357872776b9e7acf9e419476db0675d15df0d998347bc3e016a9fadecddc7ddb
2026-03-09 04:47:05.018 DEBUG [src.node.docker_mananger] Container started with ID 357872776b9e. Setting up logs at ./log/docker/node1_2026-03-09_04-47-04__3c62147d-f24e-48c1-88a1-c8e8207a7900__wakuorg_nwaku:latest.log
2026-03-09 04:47:05.019 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 4551
2026-03-09 04:47:05.019 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-09 04:47:05.391 ERROR [src.node.docker_mananger] Max retries reached for container 45ec511fb5db. Exiting log stream.
2026-03-09 04:47:06.020 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4551/health" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:06.023 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-09 04:47:06.023 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-09 04:47:06.024 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4551/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:06.027 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.126.7/tcp/4552/p2p/16Uiu2HAmHda7M2n6cP6poC4AoWuMPiRN1g3UtC4zXWoBBFUH83Bu","/ip4/172.18.126.7/tcp/4553/ws/p2p/16Uiu2HAmHda7M2n6cP6poC4AoWuMPiRN1g3UtC4zXWoBBFUH83Bu"],"enrUri":"enr:-L24QDhOWn3F5DLYtA_35lgLh0O3QUTS9Zq5B-TphL-KyFYeU5twudhhEWKT6Q7NWEzzWvTrpq6L9e7hmgGhGjQSUhwCgmlkgnY0gmlwhKwSfgeKbXVsdGlhZGRyc5YACASsEn4HBhHIAAoErBJ-BwYRyd0DgnJzhQACAQAAiXNlY3AyNTZrMaEDSe1x8mo2WV0BK9qAWclrLidhEhbFeaMbK6w22qseVziDdGNwghHIg3VkcIIRyoV3YWt1MgU"}'
2026-03-09 04:47:06.028 INFO [src.node.waku_node] REST service is ready !!
2026-03-09 04:47:06.036 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-09 04:47:06.036 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-09_04-47-04__3c62147d-f24e-48c1-88a1-c8e8207a7900__wakuorg_nwaku:latest.log
2026-03-09 04:47:06.036 DEBUG [src.node.waku_node] Starting Node...
2026-03-09 04:47:06.036 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-09 04:47:06.038 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-09 04:47:06.038 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.192.102
2026-03-09 04:47:06.038 DEBUG [src.node.docker_mananger] Generated ports ['32426', '32427', '32428', '32429', '32430']
2026-03-09 04:47:06.038 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-09 04:47:06.038 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-09 04:47:06.038 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-09 04:47:06.039 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-09 04:47:06.039 DEBUG [src.node.waku_node] Using volumes []
2026-03-09 04:47:06.039 DEBUG [src.node.docker_mananger] docker run -i -t -p 32426:32426 -p 32427:32427 -p 32428:32428 -p 32429:32429 -p 32430:32430 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=32428 --rest-port=32426 --tcp-port=32427 --discv5-udp-port=32429 --rest-address=0.0.0.0 --nat=extip:172.18.192.102 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=50e3b58eb1fffc7767aaf5cdd9bbc491d3f4123d0df2d1cdea43d2cd70f0c3ba --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=32430 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QDhOWn3F5DLYtA_35lgLh0O3QUTS9Zq5B-TphL-KyFYeU5twudhhEWKT6Q7NWEzzWvTrpq6L9e7hmgGhGjQSUhwCgmlkgnY0gmlwhKwSfgeKbXVsdGlhZGRyc5YACASsEn4HBhHIAAoErBJ-BwYRyd0DgnJzhQACAQAAiXNlY3AyNTZrMaEDSe1x8mo2WV0BK9qAWclrLidhEhbFeaMbK6w22qseVziDdGNwghHIg3VkcIIRyoV3YWt1MgU
2026-03-09 04:47:06.235 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.192.102 waku 385fc2d4d1790d817536d1c68c66ddc906539291d18126ad8729341c76f3bb68
2026-03-09 04:47:06.272 DEBUG [src.node.docker_mananger] Container started with ID 385fc2d4d179. Setting up logs at ./log/docker/node2_2026-03-09_04-47-04__3c62147d-f24e-48c1-88a1-c8e8207a7900__wakuorg_nwaku:latest.log
2026-03-09 04:47:06.272 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 32426
2026-03-09 04:47:06.272 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-09 04:47:07.273 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32426/health" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:07.275 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-09 04:47:07.275 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-09 04:47:07.276 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32426/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:07.282 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.192.102/tcp/32427/p2p/16Uiu2HAmTU75nX9frX6M73Pa2pngVnn4xzj7BvBwQysHc7Beu9f6","/ip4/172.18.192.102/tcp/32428/ws/p2p/16Uiu2HAmTU75nX9frX6M73Pa2pngVnn4xzj7BvBwQysHc7Beu9f6"],"enrUri":"enr:-L24QCJEr60zKekKQd__jOHfXf7zyfmW-DmwoOUOZg8l4FvtCSAGQYVJU7FuTFjIo5-KIhm0emyDqXC1l8Q8AKEAlGgCgmlkgnY0gmlwhKwSwGaKbXVsdGlhZGRyc5YACASsEsBmBn6rAAoErBLAZgZ-rN0DgnJzhQACAQAAiXNlY3AyNTZrMaED3BWHFTToGB3th3q_PKeF0BOaTzg95KY5VVp-_qcmznGDdGNwgn6rg3VkcIJ-rYV3YWt1MgE"}'
2026-03-09 04:47:07.282 INFO [src.node.waku_node] REST service is ready !!
2026-03-09 04:47:07.283 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32426/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.126.7/tcp/4552/p2p/16Uiu2HAmHda7M2n6cP6poC4AoWuMPiRN1g3UtC4zXWoBBFUH83Bu"]'
2026-03-09 04:47:07.314 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-09 04:47:07.315 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:4551/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-09 04:47:07.320 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-09 04:47:07.321 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32426/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-09 04:47:07.324 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-09 04:47:07.324 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:4551/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-09 04:47:07.333 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-09 04:47:07.333 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-09 04:47:07.433 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-09 04:47:07.434 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4551/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:07.436 ERROR [src.node.api_clients.base_client] HTTP error occurred: 404 Client Error: Not Found for url: http://127.0.0.1:4551/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1. Response content: b''
2026-03-09 04:47:07.481 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-09 04:47:07.483 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-09 04:47:07.483 DEBUG [src.node.waku_node] Stopping container with id 357872776b9e
2026-03-09 04:47:08.114 DEBUG [src.node.waku_node] Container stopped.
2026-03-09 04:47:08.115 DEBUG [src.node.waku_node] Stopping container with id 385fc2d4d179
2026-03-09 04:47:08.654 DEBUG [src.node.waku_node] Container stopped.
2026-03-09 04:47:08.657 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-09 04:47:08.691 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-09 04:47:08.718 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-09 04:47:08.719 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-09 04:47:08.719 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-09 04:47:08.719 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-09_04-47-04__3c62147d-f24e-48c1-88a1-c8e8207a7900__wakuorg_nwaku:latest.log
2026-03-09 04:47:08.720 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-09_04-47-04__3c62147d-f24e-48c1-88a1-c8e8207a7900__wakuorg_nwaku:latest.log
2026-03-09 04:47:08.725 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-09 04:47:08.726 DEBUG [tests.conftest] Running test: test_cant_publish_on_not_subscribed_shard with id: 2026-03-09_04-47-08__33a2ad6d-d6c3-43f0-a2e5-b118fe3c64fe
2026-03-09 04:47:08.727 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-09 04:47:08.727 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-09 04:47:08.727 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-09 04:47:08.736 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-09 04:47:08.736 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-09_04-47-08__33a2ad6d-d6c3-43f0-a2e5-b118fe3c64fe__wakuorg_nwaku:latest.log
2026-03-09 04:47:08.736 DEBUG [src.node.waku_node] Starting Node...
2026-03-09 04:47:08.736 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-09 04:47:08.738 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-09 04:47:08.738 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.192.142
2026-03-09 04:47:08.738 DEBUG [src.node.docker_mananger] Generated ports ['57929', '57930', '57931', '57932', '57933']
2026-03-09 04:47:08.738 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-09 04:47:08.738 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-09 04:47:08.738 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-09 04:47:08.739 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-09 04:47:08.739 DEBUG [src.node.waku_node] Using volumes []
2026-03-09 04:47:08.739 DEBUG [src.node.docker_mananger] docker run -i -t -p 57929:57929 -p 57930:57930 -p 57931:57931 -p 57932:57932 -p 57933:57933 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=57931 --rest-port=57929 --tcp-port=57930 --discv5-udp-port=57932 --rest-address=0.0.0.0 --nat=extip:172.18.192.142 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=289be0f34925102be87c0ec2aa8463f55dd3dd8ac83f09ed8d9eae1ebbf04cca --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=57933 --metrics-logging=true --relay=true --filter=true
2026-03-09 04:47:08.932 ERROR [src.node.docker_mananger] Max retries reached for container 357872776b9e. Exiting log stream.
2026-03-09 04:47:08.933 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.192.142 waku 5148990187ec3b012b3499376eb4acd748a00e0c88b2783ff8972d1d21b7581e
2026-03-09 04:47:08.970 DEBUG [src.node.docker_mananger] Container started with ID 5148990187ec. Setting up logs at ./log/docker/node1_2026-03-09_04-47-08__33a2ad6d-d6c3-43f0-a2e5-b118fe3c64fe__wakuorg_nwaku:latest.log
2026-03-09 04:47:08.970 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 57929
2026-03-09 04:47:08.971 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-09 04:47:09.468 ERROR [src.node.docker_mananger] Max retries reached for container 385fc2d4d179. Exiting log stream.
2026-03-09 04:47:09.972 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57929/health" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:09.976 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-09 04:47:09.977 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-09 04:47:09.979 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57929/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:09.983 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.192.142/tcp/57930/p2p/16Uiu2HAkvQWi8Dwq1DvVqRKr414DjgQhWMnuPiLefQAUG15twW6c","/ip4/172.18.192.142/tcp/57931/ws/p2p/16Uiu2HAkvQWi8Dwq1DvVqRKr414DjgQhWMnuPiLefQAUG15twW6c"],"enrUri":"enr:-L24QPkZj9-IVNWKyY0nonZ7vZA2rgKOHxbYh2h2gLHcOKBbYww08fRgXfHMQ3AtojlPaPhzYUnp_aNlrbbLk0snwwICgmlkgnY0gmlwhKwSwI6KbXVsdGlhZGRyc5YACASsEsCOBuJKAAoErBLAjgbiS90DgnJzhQACAQAAiXNlY3AyNTZrMaECDo-f1Hdg4jbwCOjCREhRKFQtOA119F9oqYp1Bsv_8fmDdGNwguJKg3VkcILiTIV3YWt1MgU"}'
2026-03-09 04:47:09.983 INFO [src.node.waku_node] REST service is ready !!
2026-03-09 04:47:09.990 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-09 04:47:09.990 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-09_04-47-08__33a2ad6d-d6c3-43f0-a2e5-b118fe3c64fe__wakuorg_nwaku:latest.log
2026-03-09 04:47:09.990 DEBUG [src.node.waku_node] Starting Node...
2026-03-09 04:47:09.990 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-09 04:47:09.992 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-09 04:47:09.992 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.88.49
2026-03-09 04:47:09.992 DEBUG [src.node.docker_mananger] Generated ports ['56290', '56291', '56292', '56293', '56294']
2026-03-09 04:47:09.992 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-09 04:47:09.992 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-09 04:47:09.992 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-09 04:47:09.992 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-09 04:47:09.993 DEBUG [src.node.waku_node] Using volumes []
2026-03-09 04:47:09.993 DEBUG [src.node.docker_mananger] docker run -i -t -p 56290:56290 -p 56291:56291 -p 56292:56292 -p 56293:56293 -p 56294:56294 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=56292 --rest-port=56290 --tcp-port=56291 --discv5-udp-port=56293 --rest-address=0.0.0.0 --nat=extip:172.18.88.49 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=d27d9e5efec2aab09b168fb16cb79a6aad1fe89eead51fc73deb883cfad12eab --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=56294 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPkZj9-IVNWKyY0nonZ7vZA2rgKOHxbYh2h2gLHcOKBbYww08fRgXfHMQ3AtojlPaPhzYUnp_aNlrbbLk0snwwICgmlkgnY0gmlwhKwSwI6KbXVsdGlhZGRyc5YACASsEsCOBuJKAAoErBLAjgbiS90DgnJzhQACAQAAiXNlY3AyNTZrMaECDo-f1Hdg4jbwCOjCREhRKFQtOA119F9oqYp1Bsv_8fmDdGNwguJKg3VkcILiTIV3YWt1MgU
2026-03-09 04:47:10.188 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.88.49 waku d7dbfa788653c90e121d5e840fb859c3dcde1d909237602ccb67b03fa6972095
2026-03-09 04:47:10.222 DEBUG [src.node.docker_mananger] Container started with ID d7dbfa788653. Setting up logs at ./log/docker/node2_2026-03-09_04-47-08__33a2ad6d-d6c3-43f0-a2e5-b118fe3c64fe__wakuorg_nwaku:latest.log
2026-03-09 04:47:10.222 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 56290
2026-03-09 04:47:10.222 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-09 04:47:11.224 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56290/health" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:11.229 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-09 04:47:11.329 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56290/health" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:11.332 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-09 04:47:11.432 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56290/health" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:11.434 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-09 04:47:11.535 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56290/health" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:11.537 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-09 04:47:11.638 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56290/health" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:11.640 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-09 04:47:11.741 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56290/health" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:11.751 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-09 04:47:11.851 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56290/health" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:11.854 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-09 04:47:11.854 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-09 04:47:11.854 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56290/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:11.857 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.88.49/tcp/56291/p2p/16Uiu2HAmNnwD2AwHyY3BvpWAEcJeMbQ3xQ8g26UWqDyHARdiya8M","/ip4/172.18.88.49/tcp/56292/ws/p2p/16Uiu2HAmNnwD2AwHyY3BvpWAEcJeMbQ3xQ8g26UWqDyHARdiya8M"],"enrUri":"enr:-L24QNBGW4NvdX4ICBEoS4Qf-7OKPk7vPmE4IBcw1PGSL_HKStBXLCgq0HjnP69lH__nDrOEu9VazcZDFWMMGLmR29kCgmlkgnY0gmlwhKwSWDGKbXVsdGlhZGRyc5YACASsElgxBtvjAAoErBJYMQbb5N0DgnJzhQACAQAAiXNlY3AyNTZrMaEDlp32YXctKyTZsyc_A6ToOItQD6lYtbkjHsnpe9G0op6DdGNwgtvjg3VkcILb5YV3YWt1MgE"}'
2026-03-09 04:47:11.857 INFO [src.node.waku_node] REST service is ready !!
2026-03-09 04:47:11.857 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56290/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.192.142/tcp/57930/p2p/16Uiu2HAkvQWi8Dwq1DvVqRKr414DjgQhWMnuPiLefQAUG15twW6c"]'
2026-03-09 04:47:11.877 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-09 04:47:11.882 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57929/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-09 04:47:11.887 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-09 04:47:11.888 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56290/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-09 04:47:11.894 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-09 04:47:11.895 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57929/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-09 04:47:11.907 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-09 04:47:11.909 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-09 04:47:12.010 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-09 04:47:12.011 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57929/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-09 04:47:12.013 ERROR [src.node.api_clients.base_client] HTTP error occurred: 404 Client Error: Not Found for url: http://127.0.0.1:57929/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1. Response content: b''
2026-03-09 04:47:12.057 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-09 04:47:12.058 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-09 04:47:12.058 DEBUG [src.node.waku_node] Stopping container with id 5148990187ec
2026-03-09 04:47:12.643 DEBUG [src.node.waku_node] Container stopped.
2026-03-09 04:47:12.643 DEBUG [src.node.waku_node] Stopping container with id d7dbfa788653
2026-03-09 04:47:13.214 DEBUG [src.node.waku_node] Container stopped.
2026-03-09 04:47:13.215 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-09 04:47:13.245 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-09 04:47:13.275 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-09 04:47:13.278 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-09 04:47:13.278 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-09 04:47:13.279 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-09_04-47-08__33a2ad6d-d6c3-43f0-a2e5-b118fe3c64fe__wakuorg_nwaku:latest.log
2026-03-09 04:47:13.284 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-09_04-47-08__33a2ad6d-d6c3-43f0-a2e5-b118fe3c64fe__wakuorg_nwaku:latest.log