187 lines
33 KiB
Plaintext

2026-03-07 04:33:28.991 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-07 04:33:28.991 DEBUG [tests.conftest] Running test: test_publish_on_multiple_pubsub_topics_and_only_after_fetch_them with id: 2026-03-07_04-33-28__bae6829e-374d-4ee8-9cc5-2f3a8ddb988d
2026-03-07 04:33:28.991 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-07 04:33:28.992 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-07 04:33:28.992 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-07 04:33:28.999 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-07 04:33:28.999 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-07_04-33-28__bae6829e-374d-4ee8-9cc5-2f3a8ddb988d__wakuorg_nwaku:latest.log
2026-03-07 04:33:29.000 DEBUG [src.node.waku_node] Starting Node...
2026-03-07 04:33:29.000 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-07 04:33:29.001 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-07 04:33:29.001 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.8.155
2026-03-07 04:33:29.001 DEBUG [src.node.docker_mananger] Generated ports ['12597', '12598', '12599', '12600', '12601']
2026-03-07 04:33:29.002 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-07 04:33:29.002 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-07 04:33:29.002 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-07 04:33:29.002 DEBUG [src.node.waku_node] Using volumes []
2026-03-07 04:33:29.002 DEBUG [src.node.docker_mananger] docker run -i -t -p 12597:12597 -p 12598:12598 -p 12599:12599 -p 12600:12600 -p 12601:12601 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=12599 --rest-port=12597 --tcp-port=12598 --discv5-udp-port=12600 --rest-address=0.0.0.0 --nat=extip:172.18.8.155 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=fef2a7fef0c60c4eb93272bf5a5a28d4e1aad1c1aedcd71dea5d91ef51518c1d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=12601 --metrics-logging=true --relay=true --filter=true
2026-03-07 04:33:29.189 ERROR [src.node.docker_mananger] Max retries reached for container 9d3052d6dc29. Exiting log stream.
2026-03-07 04:33:29.200 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.8.155 waku 2f18b47b59205428844d5b173251601ea28ab49bc89552a37d58164ab07b684c
2026-03-07 04:33:29.237 DEBUG [src.node.docker_mananger] Container started with ID 2f18b47b5920. Setting up logs at ./log/docker/node1_2026-03-07_04-33-28__bae6829e-374d-4ee8-9cc5-2f3a8ddb988d__wakuorg_nwaku:latest.log
2026-03-07 04:33:29.238 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 12597
2026-03-07 04:33:29.238 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-07 04:33:29.740 ERROR [src.node.docker_mananger] Max retries reached for container f502535722d7. Exiting log stream.
2026-03-07 04:33:30.238 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:12597/health" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:30.255 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-07 04:33:30.356 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:12597/health" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:30.358 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-07 04:33:30.358 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-07 04:33:30.359 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:12597/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:30.361 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.8.155/tcp/12598/p2p/16Uiu2HAmNmTNn6WwvhGo8CwEw1rPmyWKTBoLvgFjPcuKDBW7F6xh","/ip4/172.18.8.155/tcp/12599/ws/p2p/16Uiu2HAmNmTNn6WwvhGo8CwEw1rPmyWKTBoLvgFjPcuKDBW7F6xh"],"enrUri":"enr:-L24QEY14_Imim-WXwk6GeRJIXsYHYHVha3k9WsebRTfYo4hfnT59whivKZT1eAcduc_DjfO__OMKO4WO76sBzFVg7sCgmlkgnY0gmlwhKwSCJuKbXVsdGlhZGRyc5YACASsEgibBjE2AAoErBIImwYxN90DgnJzhQACAQAAiXNlY3AyNTZrMaEDljzpgYDxXOaFeklszvfgtcek-3yEodMfv3pRsjOoMIKDdGNwgjE2g3VkcIIxOIV3YWt1MgU"}'
2026-03-07 04:33:30.361 INFO [src.node.waku_node] REST service is ready !!
2026-03-07 04:33:30.370 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-07 04:33:30.371 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-07_04-33-28__bae6829e-374d-4ee8-9cc5-2f3a8ddb988d__wakuorg_nwaku:latest.log
2026-03-07 04:33:30.371 DEBUG [src.node.waku_node] Starting Node...
2026-03-07 04:33:30.371 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-07 04:33:30.373 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-07 04:33:30.373 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.207.167
2026-03-07 04:33:30.373 DEBUG [src.node.docker_mananger] Generated ports ['29538', '29539', '29540', '29541', '29542']
2026-03-07 04:33:30.373 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-07 04:33:30.374 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-07 04:33:30.374 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-07 04:33:30.374 DEBUG [src.node.waku_node] Using volumes []
2026-03-07 04:33:30.374 DEBUG [src.node.docker_mananger] docker run -i -t -p 29538:29538 -p 29539:29539 -p 29540:29540 -p 29541:29541 -p 29542:29542 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=29540 --rest-port=29538 --tcp-port=29539 --discv5-udp-port=29541 --rest-address=0.0.0.0 --nat=extip:172.18.207.167 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=3e3ac1e6d80e4da4efed3ffea1ce2ea6eda20beb1feecfb750c8ced8dcb8830d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=29542 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QEY14_Imim-WXwk6GeRJIXsYHYHVha3k9WsebRTfYo4hfnT59whivKZT1eAcduc_DjfO__OMKO4WO76sBzFVg7sCgmlkgnY0gmlwhKwSCJuKbXVsdGlhZGRyc5YACASsEgibBjE2AAoErBIImwYxN90DgnJzhQACAQAAiXNlY3AyNTZrMaEDljzpgYDxXOaFeklszvfgtcek-3yEodMfv3pRsjOoMIKDdGNwgjE2g3VkcIIxOIV3YWt1MgU
2026-03-07 04:33:30.581 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.207.167 waku b2599b8f31c9afb4b33efba7180536c0ba0c2d85892c7944227208687a646d95
2026-03-07 04:33:30.615 DEBUG [src.node.docker_mananger] Container started with ID b2599b8f31c9. Setting up logs at ./log/docker/node2_2026-03-07_04-33-28__bae6829e-374d-4ee8-9cc5-2f3a8ddb988d__wakuorg_nwaku:latest.log
2026-03-07 04:33:30.616 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 29538
2026-03-07 04:33:30.616 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-07 04:33:31.616 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29538/health" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:31.619 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-07 04:33:31.619 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-07 04:33:31.620 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29538/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:31.622 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.207.167/tcp/29539/p2p/16Uiu2HAkue24AR5voug2Jysk4YNWvE6kE3qv3Khnw21mrftSBaAm","/ip4/172.18.207.167/tcp/29540/ws/p2p/16Uiu2HAkue24AR5voug2Jysk4YNWvE6kE3qv3Khnw21mrftSBaAm"],"enrUri":"enr:-L24QI1o4HBCIIsmoCuRELZuD1UMd-yvpdFMszvHnIL15pPqZoGj9yZjO3ph6I8oWaQAPKabVM8tMIc1XlAeiueFUx8CgmlkgnY0gmlwhKwSz6eKbXVsdGlhZGRyc5YACASsEs-nBnNjAAoErBLPpwZzZN0DgnJzhQACAQAAiXNlY3AyNTZrMaECAymrQWMYMjORVO69T2MnboKxns3u38Zt_j67NPSlbtqDdGNwgnNjg3VkcIJzZYV3YWt1MgE"}'
2026-03-07 04:33:31.623 INFO [src.node.waku_node] REST service is ready !!
2026-03-07 04:33:31.623 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29538/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.8.155/tcp/12598/p2p/16Uiu2HAmNmTNn6WwvhGo8CwEw1rPmyWKTBoLvgFjPcuKDBW7F6xh"]'
2026-03-07 04:33:31.667 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:31.668 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12597/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-07 04:33:31.672 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:31.672 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29538/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-07 04:33:31.675 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:31.676 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12597/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzA=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:31.685 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:31.686 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12597/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzE=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:31.696 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:31.696 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12597/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzI=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:31.709 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:31.711 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12597/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F3" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzM=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:31.723 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:31.726 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12597/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F4" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzQ=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:31.739 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:31.741 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12597/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F5" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzU=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:31.753 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:31.756 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12597/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F6" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzY=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:31.767 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:31.769 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12597/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F7" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzc=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:31.782 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:31.783 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-07 04:33:31.883 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29538/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:31.885 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"L3dha3UvMi9ycy8yLzA=","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772858011676021338,"ephemeral":false,"proof":""}]'
2026-03-07 04:33:31.886 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29538/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:31.888 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"L3dha3UvMi9ycy8yLzE=","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772858011686731034,"ephemeral":false,"proof":""}]'
2026-03-07 04:33:31.889 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29538/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:31.892 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"L3dha3UvMi9ycy8yLzI=","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772858011696561504,"ephemeral":false,"proof":""}]'
2026-03-07 04:33:31.892 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29538/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F3" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:31.894 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772858011659239033,"ephemeral":false,"proof":""},{"payload":"L3dha3UvMi9ycy8yLzM=","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772858011711611831,"ephemeral":false,"proof":""}]'
2026-03-07 04:33:31.937 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-07 04:33:31.940 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-07 04:33:31.940 DEBUG [src.node.waku_node] Stopping container with id 2f18b47b5920
2026-03-07 04:33:32.518 DEBUG [src.node.waku_node] Container stopped.
2026-03-07 04:33:32.520 DEBUG [src.node.waku_node] Stopping container with id b2599b8f31c9
2026-03-07 04:33:33.107 DEBUG [src.node.waku_node] Container stopped.
2026-03-07 04:33:33.109 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-07 04:33:33.148 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-07 04:33:33.189 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-07 04:33:33.190 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-07 04:33:33.191 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-07 04:33:33.194 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-07_04-33-28__bae6829e-374d-4ee8-9cc5-2f3a8ddb988d__wakuorg_nwaku:latest.log
2026-03-07 04:33:33.199 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-07_04-33-28__bae6829e-374d-4ee8-9cc5-2f3a8ddb988d__wakuorg_nwaku:latest.log
2026-03-07 04:33:33.214 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-07 04:33:33.218 DEBUG [tests.conftest] Running test: test_publish_on_multiple_pubsub_topics_and_only_after_fetch_them with id: 2026-03-07_04-33-33__c188c062-08e4-4b25-923a-5671fe740e86
2026-03-07 04:33:33.218 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-07 04:33:33.219 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-07 04:33:33.219 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-07 04:33:33.233 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-07 04:33:33.234 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-07_04-33-33__c188c062-08e4-4b25-923a-5671fe740e86__wakuorg_nwaku:latest.log
2026-03-07 04:33:33.234 DEBUG [src.node.waku_node] Starting Node...
2026-03-07 04:33:33.234 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-07 04:33:33.236 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-07 04:33:33.237 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.88.47
2026-03-07 04:33:33.237 DEBUG [src.node.docker_mananger] Generated ports ['14306', '14307', '14308', '14309', '14310']
2026-03-07 04:33:33.237 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-07 04:33:33.238 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-07 04:33:33.239 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-07 04:33:33.239 DEBUG [src.node.waku_node] Using volumes []
2026-03-07 04:33:33.240 DEBUG [src.node.docker_mananger] docker run -i -t -p 14306:14306 -p 14307:14307 -p 14308:14308 -p 14309:14309 -p 14310:14310 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=14308 --rest-port=14306 --tcp-port=14307 --discv5-udp-port=14309 --rest-address=0.0.0.0 --nat=extip:172.18.88.47 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=cc74dcdcdd05c6aaceaaa6e5ff8d8ea0993e38cfda0afbdf1a6aa5aaf9a0ab95 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=14310 --metrics-logging=true --relay=true --filter=true
2026-03-07 04:33:33.397 ERROR [src.node.docker_mananger] Max retries reached for container 2f18b47b5920. Exiting log stream.
2026-03-07 04:33:33.460 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.88.47 waku 78d7d482a9e4f7a38265bfff63e57aa37c9ffa316b091e662b26ea19b05c5872
2026-03-07 04:33:33.495 DEBUG [src.node.docker_mananger] Container started with ID 78d7d482a9e4. Setting up logs at ./log/docker/node1_2026-03-07_04-33-33__c188c062-08e4-4b25-923a-5671fe740e86__wakuorg_nwaku:latest.log
2026-03-07 04:33:33.496 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 14306
2026-03-07 04:33:33.496 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-07 04:33:34.173 ERROR [src.node.docker_mananger] Max retries reached for container b2599b8f31c9. Exiting log stream.
2026-03-07 04:33:34.496 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:14306/health" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:34.499 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-07 04:33:34.499 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-07 04:33:34.500 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:14306/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:34.502 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.88.47/tcp/14307/p2p/16Uiu2HAm25Z2BrX5mcUiXYLogAcZpFMcqpfR8N5by1XunZHY5e2n","/ip4/172.18.88.47/tcp/14308/ws/p2p/16Uiu2HAm25Z2BrX5mcUiXYLogAcZpFMcqpfR8N5by1XunZHY5e2n"],"enrUri":"enr:-L24QGpAsKFt5BnqOxgqkxzDbhR9fIQc00AcyXnxnrcIjFKQDFdtZnSXPaOTIf7H882ViqcM5KxpMNRknnhDwotW2BICgmlkgnY0gmlwhKwSWC-KbXVsdGlhZGRyc5YACASsElgvBjfjAAoErBJYLwY35N0DgnJzhQACAQAAiXNlY3AyNTZrMaECYtpS9WVjU9-WxkUrw-c95FjyYUZY6rGrr71C_5RL_quDdGNwgjfjg3VkcII35YV3YWt1MgU"}'
2026-03-07 04:33:34.503 INFO [src.node.waku_node] REST service is ready !!
2026-03-07 04:33:34.511 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-07 04:33:34.511 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-07_04-33-33__c188c062-08e4-4b25-923a-5671fe740e86__wakuorg_nwaku:latest.log
2026-03-07 04:33:34.511 DEBUG [src.node.waku_node] Starting Node...
2026-03-07 04:33:34.512 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-07 04:33:34.513 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-07 04:33:34.514 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.30.118
2026-03-07 04:33:34.514 DEBUG [src.node.docker_mananger] Generated ports ['8677', '8678', '8679', '8680', '8681']
2026-03-07 04:33:34.514 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-07 04:33:34.515 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-07 04:33:34.515 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-07 04:33:34.515 DEBUG [src.node.waku_node] Using volumes []
2026-03-07 04:33:34.515 DEBUG [src.node.docker_mananger] docker run -i -t -p 8677:8677 -p 8678:8678 -p 8679:8679 -p 8680:8680 -p 8681:8681 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=8679 --rest-port=8677 --tcp-port=8678 --discv5-udp-port=8680 --rest-address=0.0.0.0 --nat=extip:172.18.30.118 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=11be4f1879674d62c59e095c1341cbe9cd854235cfe95accee0c64b4d18f557b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=8681 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QGpAsKFt5BnqOxgqkxzDbhR9fIQc00AcyXnxnrcIjFKQDFdtZnSXPaOTIf7H882ViqcM5KxpMNRknnhDwotW2BICgmlkgnY0gmlwhKwSWC-KbXVsdGlhZGRyc5YACASsElgvBjfjAAoErBJYLwY35N0DgnJzhQACAQAAiXNlY3AyNTZrMaECYtpS9WVjU9-WxkUrw-c95FjyYUZY6rGrr71C_5RL_quDdGNwgjfjg3VkcII35YV3YWt1MgU
2026-03-07 04:33:34.709 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.30.118 waku e05018e5b7a4b835579361ceda300986e60456140a0788e2af7d21685ecbdc0f
2026-03-07 04:33:34.742 DEBUG [src.node.docker_mananger] Container started with ID e05018e5b7a4. Setting up logs at ./log/docker/node2_2026-03-07_04-33-33__c188c062-08e4-4b25-923a-5671fe740e86__wakuorg_nwaku:latest.log
2026-03-07 04:33:34.742 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 8677
2026-03-07 04:33:34.742 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-07 04:33:35.744 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8677/health" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:35.747 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-07 04:33:35.747 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-07 04:33:35.747 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8677/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:35.749 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.30.118/tcp/8678/p2p/16Uiu2HAmB26gtJXXGLixM7TN91qDupWAsQBhxPhVL3T8uLKJZ4yK","/ip4/172.18.30.118/tcp/8679/ws/p2p/16Uiu2HAmB26gtJXXGLixM7TN91qDupWAsQBhxPhVL3T8uLKJZ4yK"],"enrUri":"enr:-L24QE5ttWJjEyq-jB2Ks4NefJpzMX2degXzGOJwBrzBTASJIAtrBWa4YW-0fwneAVJHGCHqbYGj4AaEesYNdyxAeAwCgmlkgnY0gmlwhKwSHnaKbXVsdGlhZGRyc5YACASsEh52BiHmAAoErBIedgYh590DgnJzhQACAQAAiXNlY3AyNTZrMaEC57D9R2hbOeA02TMFSMO8pbWZgp_o9l3bB_3B7AmEAD6DdGNwgiHmg3VkcIIh6IV3YWt1MgE"}'
2026-03-07 04:33:35.750 INFO [src.node.waku_node] REST service is ready !!
2026-03-07 04:33:35.750 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:8677/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.88.47/tcp/14307/p2p/16Uiu2HAm25Z2BrX5mcUiXYLogAcZpFMcqpfR8N5by1XunZHY5e2n"]'
2026-03-07 04:33:35.771 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:35.773 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:14306/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-07 04:33:35.782 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:35.782 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:8677/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-07 04:33:35.788 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:35.789 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:14306/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzA=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:35.800 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:35.802 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:14306/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzE=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:35.813 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:35.814 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:14306/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzI=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:35.823 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:35.824 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:14306/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F3" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzM=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:35.835 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:35.836 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:14306/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F4" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzQ=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:35.845 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:35.846 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:14306/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F5" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzU=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:35.860 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:35.861 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:14306/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F6" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzY=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:35.872 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:35.873 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:14306/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F7" -H "Content-Type: application/json" -d '{"payload": "L3dha3UvMi9ycy8yLzc=", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-07 04:33:35.883 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-07 04:33:35.884 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-07 04:33:35.985 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8677/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-07 04:33:35.988 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"L3dha3UvMi9ycy8yLzA=","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772858015789167383,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772858015794117412,"ephemeral":false,"proof":""}]'
2026-03-07 04:33:35.993 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-07 04:33:35.994 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-07 04:33:35.994 DEBUG [src.node.waku_node] Stopping container with id 78d7d482a9e4
2026-03-07 04:33:36.602 DEBUG [src.node.waku_node] Container stopped.
2026-03-07 04:33:36.603 DEBUG [src.node.waku_node] Stopping container with id e05018e5b7a4
2026-03-07 04:33:37.199 DEBUG [src.node.waku_node] Container stopped.
2026-03-07 04:33:37.200 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-07 04:33:37.233 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-07 04:33:37.274 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-07 04:33:37.275 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-07 04:33:37.275 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-07 04:33:37.276 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-07_04-33-33__c188c062-08e4-4b25-923a-5671fe740e86__wakuorg_nwaku:latest.log
2026-03-07 04:33:37.277 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-07_04-33-33__c188c062-08e4-4b25-923a-5671fe740e86__wakuorg_nwaku:latest.log