161 lines
27 KiB
Plaintext

2026-03-11 04:36:56.099 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-11 04:36:56.100 DEBUG [tests.conftest] Running test: test_multiple_pubsub_topics_same_cluster with id: 2026-03-11_04-36-56__2ca27052-409a-4465-bd8b-e2d8ebee7504
2026-03-11 04:36:56.101 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-11 04:36:56.102 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-11 04:36:56.103 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-11 04:36:56.113 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:36:56.113 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-11_04-36-56__2ca27052-409a-4465-bd8b-e2d8ebee7504__wakuorg_nwaku:latest.log
2026-03-11 04:36:56.113 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:36:56.113 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:36:56.115 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:36:56.115 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.187.121
2026-03-11 04:36:56.115 DEBUG [src.node.docker_mananger] Generated ports ['26340', '26341', '26342', '26343', '26344']
2026-03-11 04:36:56.115 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-11 04:36:56.115 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-11 04:36:56.115 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:36:56.115 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:36:56.116 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:36:56.116 DEBUG [src.node.docker_mananger] docker run -i -t -p 26340:26340 -p 26341:26341 -p 26342:26342 -p 26343:26343 -p 26344:26344 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=26342 --rest-port=26340 --tcp-port=26341 --discv5-udp-port=26343 --rest-address=0.0.0.0 --nat=extip:172.18.187.121 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=c3ceacfbee4b3dbec8ebd08d48eeccc824d4fdc1bcd8437c9abb542de1cd517b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=26344 --metrics-logging=true --relay=true --filter=true
2026-03-11 04:36:56.315 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.187.121 waku da422bd1307373379a6463f6090d66c5956c7ef79ea4987c92b73b623088b6b8
2026-03-11 04:36:56.352 ERROR [src.node.docker_mananger] Max retries reached for container dde464989b3b. Exiting log stream.
2026-03-11 04:36:56.353 DEBUG [src.node.docker_mananger] Container started with ID da422bd13073. Setting up logs at ./log/docker/node1_2026-03-11_04-36-56__2ca27052-409a-4465-bd8b-e2d8ebee7504__wakuorg_nwaku:latest.log
2026-03-11 04:36:56.354 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 26340
2026-03-11 04:36:56.354 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:36:56.920 ERROR [src.node.docker_mananger] Max retries reached for container c01c63fe0800. Exiting log stream.
2026-03-11 04:36:57.354 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:26340/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:57.386 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:36:57.487 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:26340/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:57.489 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:36:57.489 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:36:57.490 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:26340/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:57.492 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.187.121/tcp/26341/p2p/16Uiu2HAm3fbcyhmygQuftM5oZHsXx1UnaQsN669uPbugqYPFm2GZ","/ip4/172.18.187.121/tcp/26342/ws/p2p/16Uiu2HAm3fbcyhmygQuftM5oZHsXx1UnaQsN669uPbugqYPFm2GZ"],"enrUri":"enr:-L24QFrBuDPWEofFLnp0i5c-lna4apsX20-N4UyGHj_tWmasRXra73UBv0Kw1jFbhVMWHMlFPD1PAcH6n9PRF_JVH9QCgmlkgnY0gmlwhKwSu3mKbXVsdGlhZGRyc5YACASsErt5BmblAAoErBK7eQZm5t0DgnJzhQACAQAAiXNlY3AyNTZrMaECem6yuQxDrjc2WBh0qRwCodk6Z_LrWqpVXrm5b0QvQuqDdGNwgmblg3VkcIJm54V3YWt1MgU"}'
2026-03-11 04:36:57.492 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:36:57.499 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:36:57.499 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-11_04-36-56__2ca27052-409a-4465-bd8b-e2d8ebee7504__wakuorg_nwaku:latest.log
2026-03-11 04:36:57.500 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:36:57.500 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:36:57.501 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:36:57.501 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.50.253
2026-03-11 04:36:57.501 DEBUG [src.node.docker_mananger] Generated ports ['48063', '48064', '48065', '48066', '48067']
2026-03-11 04:36:57.501 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-11 04:36:57.501 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-11 04:36:57.502 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:36:57.502 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:36:57.502 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:36:57.502 DEBUG [src.node.docker_mananger] docker run -i -t -p 48063:48063 -p 48064:48064 -p 48065:48065 -p 48066:48066 -p 48067:48067 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=48065 --rest-port=48063 --tcp-port=48064 --discv5-udp-port=48066 --rest-address=0.0.0.0 --nat=extip:172.18.50.253 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=af75755f8ccfe1fff8ccbcfb2358cfdfcfdf7f812fa04c4e541eb7f6764cdcf9 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=48067 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QFrBuDPWEofFLnp0i5c-lna4apsX20-N4UyGHj_tWmasRXra73UBv0Kw1jFbhVMWHMlFPD1PAcH6n9PRF_JVH9QCgmlkgnY0gmlwhKwSu3mKbXVsdGlhZGRyc5YACASsErt5BmblAAoErBK7eQZm5t0DgnJzhQACAQAAiXNlY3AyNTZrMaECem6yuQxDrjc2WBh0qRwCodk6Z_LrWqpVXrm5b0QvQuqDdGNwgmblg3VkcIJm54V3YWt1MgU
2026-03-11 04:36:57.706 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.50.253 waku af7fac6400d9da2fabbd74fa3d6c9d38ee8db05673ba1ba22adae92c417d441a
2026-03-11 04:36:57.745 DEBUG [src.node.docker_mananger] Container started with ID af7fac6400d9. Setting up logs at ./log/docker/node2_2026-03-11_04-36-56__2ca27052-409a-4465-bd8b-e2d8ebee7504__wakuorg_nwaku:latest.log
2026-03-11 04:36:57.745 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 48063
2026-03-11 04:36:57.746 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:36:58.746 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:48063/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:58.749 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:36:58.749 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:36:58.749 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:48063/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:58.752 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.50.253/tcp/48064/p2p/16Uiu2HAmD6LyRGCYaZDz3RUjSQQs1vSEeYBvyumR5B795BNruKh9","/ip4/172.18.50.253/tcp/48065/ws/p2p/16Uiu2HAmD6LyRGCYaZDz3RUjSQQs1vSEeYBvyumR5B795BNruKh9"],"enrUri":"enr:-L24QBaNE_fhaix5tzuaojr-SJa0941v4bwMRsvAkR61R1RwX8jHSWGidGOo8Cxj4UmbWBBeZXPd28W--0-m5qA7scgCgmlkgnY0gmlwhKwSMv2KbXVsdGlhZGRyc5YACASsEjL9BrvAAAoErBIy_Qa7wd0DgnJzhQACAQAAiXNlY3AyNTZrMaEDBn7WsMRMrDhN4L7gkIaVQeY8dqARpEopWcPunqWoYHCDdGNwgrvAg3VkcIK7woV3YWt1MgE"}'
2026-03-11 04:36:58.752 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:36:58.752 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48063/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.187.121/tcp/26341/p2p/16Uiu2HAm3fbcyhmygQuftM5oZHsXx1UnaQsN669uPbugqYPFm2GZ"]'
2026-03-11 04:36:58.780 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:58.780 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:26340/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-11 04:36:58.796 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:58.797 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48063/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-11 04:36:58.802 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:58.804 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:26340/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:36:58.821 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:36:58.822 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-11 04:36:58.924 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-11 04:36:58.924 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:26340/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:36:58.927 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773203817984038907,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773203818804039567,"ephemeral":false,"proof":""}]'
2026-03-11 04:36:58.935 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-11 04:36:58.936 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-11 04:36:58.936 DEBUG [src.node.waku_node] Stopping container with id da422bd13073
2026-03-11 04:36:59.532 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:36:59.532 DEBUG [src.node.waku_node] Stopping container with id af7fac6400d9
2026-03-11 04:37:00.079 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:37:00.082 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-11 04:37:00.112 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:37:00.145 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:37:00.147 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-11 04:37:00.147 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-11 04:37:00.152 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-11_04-36-56__2ca27052-409a-4465-bd8b-e2d8ebee7504__wakuorg_nwaku:latest.log
2026-03-11 04:37:00.154 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-11_04-36-56__2ca27052-409a-4465-bd8b-e2d8ebee7504__wakuorg_nwaku:latest.log
2026-03-11 04:37:00.169 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-11 04:37:00.170 DEBUG [tests.conftest] Running test: test_multiple_pubsub_topics_same_cluster with id: 2026-03-11_04-37-00__86116cc6-2488-40ad-99a8-0c3146c90463
2026-03-11 04:37:00.171 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-11 04:37:00.171 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-11 04:37:00.172 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-11 04:37:00.183 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:37:00.183 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-11_04-37-00__86116cc6-2488-40ad-99a8-0c3146c90463__wakuorg_nwaku:latest.log
2026-03-11 04:37:00.184 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:37:00.184 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:37:00.186 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:37:00.186 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.229.162
2026-03-11 04:37:00.187 DEBUG [src.node.docker_mananger] Generated ports ['54834', '54835', '54836', '54837', '54838']
2026-03-11 04:37:00.188 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-11 04:37:00.188 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-11 04:37:00.188 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:37:00.188 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:37:00.189 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:37:00.190 DEBUG [src.node.docker_mananger] docker run -i -t -p 54834:54834 -p 54835:54835 -p 54836:54836 -p 54837:54837 -p 54838:54838 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=54836 --rest-port=54834 --tcp-port=54835 --discv5-udp-port=54837 --rest-address=0.0.0.0 --nat=extip:172.18.229.162 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=948d2aec9ba03dbe46b2e7ec1eea0f8bb20b9eee068adef47a83853c8b20ef2f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=54838 --metrics-logging=true --relay=true --filter=true
2026-03-11 04:37:00.354 ERROR [src.node.docker_mananger] Max retries reached for container da422bd13073. Exiting log stream.
2026-03-11 04:37:00.404 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.229.162 waku 4ff872a11819d51dfde0fd69244950e5d5ce114a89048cb49e1b3f85dd7f9c7b
2026-03-11 04:37:00.439 DEBUG [src.node.docker_mananger] Container started with ID 4ff872a11819. Setting up logs at ./log/docker/node1_2026-03-11_04-37-00__86116cc6-2488-40ad-99a8-0c3146c90463__wakuorg_nwaku:latest.log
2026-03-11 04:37:00.440 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 54834
2026-03-11 04:37:00.443 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:37:01.093 ERROR [src.node.docker_mananger] Max retries reached for container af7fac6400d9. Exiting log stream.
2026-03-11 04:37:01.443 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:54834/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:37:01.446 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:37:01.446 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:37:01.446 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:54834/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:37:01.449 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.229.162/tcp/54835/p2p/16Uiu2HAm8cdnHRF28vJ34VA6UJrtfXueaWPDrn2r6FdtWz5jWBu1","/ip4/172.18.229.162/tcp/54836/ws/p2p/16Uiu2HAm8cdnHRF28vJ34VA6UJrtfXueaWPDrn2r6FdtWz5jWBu1"],"enrUri":"enr:-L24QDCpYmmLL7hd_QBBhB8Vnf1jZwkTyex16-RcEE3I5xosEVvQWRbhtaoZJgakUuo8-mxFPv9wMJQ3r8DRlHpa3cYCgmlkgnY0gmlwhKwS5aKKbXVsdGlhZGRyc5YACASsEuWiBtYzAAoErBLlogbWNN0DgnJzhQACAQAAiXNlY3AyNTZrMaECw_bVSblAaSlxJZx7s6_TcmT3Ev2E0yULpPkFK4mbYJiDdGNwgtYzg3VkcILWNYV3YWt1MgU"}'
2026-03-11 04:37:01.449 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:37:01.456 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-11 04:37:01.456 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-11_04-37-00__86116cc6-2488-40ad-99a8-0c3146c90463__wakuorg_nwaku:latest.log
2026-03-11 04:37:01.456 DEBUG [src.node.waku_node] Starting Node...
2026-03-11 04:37:01.456 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-11 04:37:01.458 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-11 04:37:01.458 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.194.44
2026-03-11 04:37:01.458 DEBUG [src.node.docker_mananger] Generated ports ['52621', '52622', '52623', '52624', '52625']
2026-03-11 04:37:01.458 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-11 04:37:01.458 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-11 04:37:01.459 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-11 04:37:01.459 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-11 04:37:01.459 DEBUG [src.node.waku_node] Using volumes []
2026-03-11 04:37:01.459 DEBUG [src.node.docker_mananger] docker run -i -t -p 52621:52621 -p 52622:52622 -p 52623:52623 -p 52624:52624 -p 52625:52625 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=52623 --rest-port=52621 --tcp-port=52622 --discv5-udp-port=52624 --rest-address=0.0.0.0 --nat=extip:172.18.194.44 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=cdc2e076aaa03f7a3cce6a2dd1daefafbab9cc3bf1cfe59b1cea3fdcb1f0ef42 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=52625 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QDCpYmmLL7hd_QBBhB8Vnf1jZwkTyex16-RcEE3I5xosEVvQWRbhtaoZJgakUuo8-mxFPv9wMJQ3r8DRlHpa3cYCgmlkgnY0gmlwhKwS5aKKbXVsdGlhZGRyc5YACASsEuWiBtYzAAoErBLlogbWNN0DgnJzhQACAQAAiXNlY3AyNTZrMaECw_bVSblAaSlxJZx7s6_TcmT3Ev2E0yULpPkFK4mbYJiDdGNwgtYzg3VkcILWNYV3YWt1MgU
2026-03-11 04:37:01.657 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.194.44 waku 65c38522831fea8040d836255a2cfbc390fe6ad825e729a5409c020ab4698cbc
2026-03-11 04:37:01.690 DEBUG [src.node.docker_mananger] Container started with ID 65c38522831f. Setting up logs at ./log/docker/node2_2026-03-11_04-37-00__86116cc6-2488-40ad-99a8-0c3146c90463__wakuorg_nwaku:latest.log
2026-03-11 04:37:01.692 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 52621
2026-03-11 04:37:01.694 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-11 04:37:02.694 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52621/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:37:02.725 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-11 04:37:02.825 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52621/health" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:37:02.828 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-11 04:37:02.828 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-11 04:37:02.828 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52621/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:37:02.831 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.194.44/tcp/52622/p2p/16Uiu2HAmLs57h74TLWnUiFUgAsJfRQZ2SgxLUrKsqndKsDctZiJN","/ip4/172.18.194.44/tcp/52623/ws/p2p/16Uiu2HAmLs57h74TLWnUiFUgAsJfRQZ2SgxLUrKsqndKsDctZiJN"],"enrUri":"enr:-L24QBpP46P6o6FHTNffK4D7uy7EGGUcvgjJ6VjOfLSDG8mmCH-iq2TJhII5qYoYP3PI1wyPR-XNorro0rgZCPBShm0CgmlkgnY0gmlwhKwSwiyKbXVsdGlhZGRyc5YACASsEsIsBs2OAAoErBLCLAbNj90DgnJzhQACAQAAiXNlY3AyNTZrMaEDefXagoNMnlUj2iZp7BV8tKh2ujriWmLduV7zPIK8OMODdGNwgs2Og3VkcILNkIV3YWt1MgE"}'
2026-03-11 04:37:02.831 INFO [src.node.waku_node] REST service is ready !!
2026-03-11 04:37:02.831 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:52621/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.229.162/tcp/54835/p2p/16Uiu2HAm8cdnHRF28vJ34VA6UJrtfXueaWPDrn2r6FdtWz5jWBu1"]'
2026-03-11 04:37:02.859 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:37:02.861 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:54834/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-11 04:37:02.866 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:37:02.867 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:52621/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-11 04:37:02.871 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:37:02.872 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:54834/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-11 04:37:02.882 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-11 04:37:02.883 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-11 04:37:02.984 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-11 04:37:02.984 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:54834/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-11 04:37:02.987 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773203822267610347,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773203822872287216,"ephemeral":false,"proof":""}]'
2026-03-11 04:37:02.995 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-11 04:37:02.996 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-11 04:37:02.996 DEBUG [src.node.waku_node] Stopping container with id 4ff872a11819
2026-03-11 04:37:03.583 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:37:03.584 DEBUG [src.node.waku_node] Stopping container with id 65c38522831f
2026-03-11 04:37:04.201 DEBUG [src.node.waku_node] Container stopped.
2026-03-11 04:37:04.204 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-11 04:37:04.261 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:37:04.287 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-11 04:37:04.287 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-11 04:37:04.287 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-11 04:37:04.288 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-11_04-37-00__86116cc6-2488-40ad-99a8-0c3146c90463__wakuorg_nwaku:latest.log
2026-03-11 04:37:04.289 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-11_04-37-00__86116cc6-2488-40ad-99a8-0c3146c90463__wakuorg_nwaku:latest.log