304 lines
55 KiB
Plaintext

2026-04-23 05:15:00.217 DEBUG [tests.conftest] Running fixture setup: test_id
2026-04-23 05:15:00.218 DEBUG [tests.conftest] Running test: test_light_push_with_missing_pubsub_topics with id: 2026-04-23_05-15-00__5ddd50f8-022e-49e5-9f91-907730b082bd
2026-04-23 05:15:00.218 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-04-23 05:15:00.218 DEBUG [src.steps.light_push] Running fixture setup: light_push_setup
2026-04-23 05:15:00.227 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 05:15:00.228 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node1_2026-04-23_05-15-00__5ddd50f8-022e-49e5-9f91-907730b082bd__wakuorg_nwaku:latest.log
2026-04-23 05:15:00.228 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 05:15:00.228 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 05:15:00.230 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 05:15:00.230 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.13.210
2026-04-23 05:15:00.230 DEBUG [src.node.docker_mananger] Generated ports ['52219', '52220', '52221', '52222', '52223']
2026-04-23 05:15:00.231 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 05:15:00.231 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 05:15:00.231 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 05:15:00.231 DEBUG [src.node.docker_mananger] docker run -i -t -p 52219:52219 -p 52220:52220 -p 52221:52221 -p 52222:52222 -p 52223:52223 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=52221 --rest-port=52219 --tcp-port=52220 --discv5-udp-port=52222 --rest-address=0.0.0.0 --nat=extip:172.18.13.210 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=605c3ba78e0d7ae80fdf03a4dfaa14ff7dbdf6e126862cbb6f5cab7d77f8cfa7 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=52223 --metrics-logging=true --lightpush=true --relay=true
2026-04-23 05:15:00.339 ERROR [src.node.docker_mananger] Max retries reached for container f150450b3214. Exiting log stream.
2026-04-23 05:15:00.434 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.13.210 waku 2474d5a775872828ec083240956cddb1029c2e2ccab97c4b245059b4b0f241f8
2026-04-23 05:15:00.474 DEBUG [src.node.docker_mananger] Container started with ID 2474d5a77587. Setting up logs at ./log/docker/receiving_node1_2026-04-23_05-15-00__5ddd50f8-022e-49e5-9f91-907730b082bd__wakuorg_nwaku:latest.log
2026-04-23 05:15:00.475 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 52219
2026-04-23 05:15:00.475 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 05:15:01.066 ERROR [src.node.docker_mananger] Max retries reached for container 5718ce6b5468. Exiting log stream.
2026-04-23 05:15:01.475 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52219/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:01.478 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 05:15:01.478 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 05:15:01.479 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52219/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:01.481 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.13.210/tcp/52220/p2p/16Uiu2HAmTgzwWbGCvi4thx5qoYYpj3W4zGzDuf5rVDeo8jv1ognU","/ip4/172.18.13.210/tcp/52221/ws/p2p/16Uiu2HAmTgzwWbGCvi4thx5qoYYpj3W4zGzDuf5rVDeo8jv1ognU"],"enrUri":"enr:-L24QDXCtwJwmFWk_J17m_eyHk__zGQCtguRU0qVROVmzlHwRlY9R2iy4qn7vLSxJuqv0D9US7afPWSIZtppFJH1BJQCgmlkgnY0gmlwhKwSDdKKbXVsdGlhZGRyc5YACASsEg3SBsv8AAoErBIN0gbL_d0DgnJzhQDGAQAAiXNlY3AyNTZrMaED32Mh9WIYX8JRRh3xfvpXHHV8aDMt29i9JuONDnJkuBmDdGNwgsv8g3VkcILL_oV3YWt1Mgk"}'
2026-04-23 05:15:01.482 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 05:15:01.489 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 05:15:01.489 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node2_2026-04-23_05-15-00__5ddd50f8-022e-49e5-9f91-907730b082bd__wakuorg_nwaku:latest.log
2026-04-23 05:15:01.489 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 05:15:01.489 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 05:15:01.491 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 05:15:01.491 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.107.155
2026-04-23 05:15:01.491 DEBUG [src.node.docker_mananger] Generated ports ['6981', '6982', '6983', '6984', '6985']
2026-04-23 05:15:01.491 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 05:15:01.492 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 05:15:01.492 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 05:15:01.492 DEBUG [src.node.docker_mananger] docker run -i -t -p 6981:6981 -p 6982:6982 -p 6983:6983 -p 6984:6984 -p 6985:6985 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=6983 --rest-port=6981 --tcp-port=6982 --discv5-udp-port=6984 --rest-address=0.0.0.0 --nat=extip:172.18.107.155 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=cb23cb0b76a7df5b09a75feae3d2b34cf84b4efe5cdfd5bcad0cfffcf134ba3b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=6985 --metrics-logging=true --lightpush=false --relay=true
2026-04-23 05:15:01.686 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.107.155 waku e42a8832e4c6f051e511c1f2f04c4044b64a43399922e7d12abef02589eea0c3
2026-04-23 05:15:01.720 DEBUG [src.node.docker_mananger] Container started with ID e42a8832e4c6. Setting up logs at ./log/docker/receiving_node2_2026-04-23_05-15-00__5ddd50f8-022e-49e5-9f91-907730b082bd__wakuorg_nwaku:latest.log
2026-04-23 05:15:01.720 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 6981
2026-04-23 05:15:01.721 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 05:15:02.721 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6981/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:02.724 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 05:15:02.725 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 05:15:02.725 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6981/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:02.728 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.107.155/tcp/6982/p2p/16Uiu2HAmEyebw3hhUS8PdSamoEBtH7A72byC9kgaUiJ4Cvr2Gs2M","/ip4/172.18.107.155/tcp/6983/ws/p2p/16Uiu2HAmEyebw3hhUS8PdSamoEBtH7A72byC9kgaUiJ4Cvr2Gs2M"],"enrUri":"enr:-L24QNo3rigEFQFwXboncDwen6KFnJ2rXBZH3IqheSCpNTuzJmEHV9aotRw8V7n--JXD89nXpGloj10gJQsrbxQQ1RkCgmlkgnY0gmlwhKwSa5uKbXVsdGlhZGRyc5YACASsEmubBhtGAAoErBJrmwYbR90DgnJzhQDGAQAAiXNlY3AyNTZrMaEDIn8UowNEud_FMjWTAVIIfGCXTPzXnew37AGiocZtEh6DdGNwghtGg3VkcIIbSIV3YWt1MgE"}'
2026-04-23 05:15:02.728 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 05:15:02.728 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6981/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.13.210/tcp/52220/p2p/16Uiu2HAmTgzwWbGCvi4thx5qoYYpj3W4zGzDuf5rVDeo8jv1ognU"]'
2026-04-23 05:15:02.763 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:02.775 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 05:15:02.776 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node1_2026-04-23_05-15-00__5ddd50f8-022e-49e5-9f91-907730b082bd__wakuorg_nwaku:latest.log
2026-04-23 05:15:02.776 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 05:15:02.776 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 05:15:02.777 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 05:15:02.777 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.87.149
2026-04-23 05:15:02.778 DEBUG [src.node.docker_mananger] Generated ports ['59079', '59080', '59081', '59082', '59083']
2026-04-23 05:15:02.778 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 05:15:02.778 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 05:15:02.778 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 05:15:02.778 DEBUG [src.node.docker_mananger] docker run -i -t -p 59079:59079 -p 59080:59080 -p 59081:59081 -p 59082:59082 -p 59083:59083 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=59081 --rest-port=59079 --tcp-port=59080 --discv5-udp-port=59082 --rest-address=0.0.0.0 --nat=extip:172.18.87.149 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=96ac9fccc0847cc6cefdcde8fa6cab60269e1be436bc42acb66e9a77d801ad7a --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=59083 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QDXCtwJwmFWk_J17m_eyHk__zGQCtguRU0qVROVmzlHwRlY9R2iy4qn7vLSxJuqv0D9US7afPWSIZtppFJH1BJQCgmlkgnY0gmlwhKwSDdKKbXVsdGlhZGRyc5YACASsEg3SBsv8AAoErBIN0gbL_d0DgnJzhQDGAQAAiXNlY3AyNTZrMaED32Mh9WIYX8JRRh3xfvpXHHV8aDMt29i9JuONDnJkuBmDdGNwgsv8g3VkcILL_oV3YWt1Mgk --lightpushnode=/ip4/172.18.13.210/tcp/52220/p2p/16Uiu2HAmTgzwWbGCvi4thx5qoYYpj3W4zGzDuf5rVDeo8jv1ognU --lightpush=true --relay=true
2026-04-23 05:15:02.987 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.87.149 waku cf7da982ff9ae652edca5079395984d03e44f5b18bd4c20972f06f2e65a7eac6
2026-04-23 05:15:03.027 DEBUG [src.node.docker_mananger] Container started with ID cf7da982ff9a. Setting up logs at ./log/docker/lightpush_node1_2026-04-23_05-15-00__5ddd50f8-022e-49e5-9f91-907730b082bd__wakuorg_nwaku:latest.log
2026-04-23 05:15:03.027 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 59079
2026-04-23 05:15:03.028 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 05:15:04.031 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59079/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:04.035 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 05:15:04.035 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 05:15:04.035 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59079/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:04.038 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.87.149/tcp/59080/p2p/16Uiu2HAmSYnQajKR4LqgimrY4UTv6CdJ6XeVpCvpB83mHzRqxbyM","/ip4/172.18.87.149/tcp/59081/ws/p2p/16Uiu2HAmSYnQajKR4LqgimrY4UTv6CdJ6XeVpCvpB83mHzRqxbyM"],"enrUri":"enr:-L24QB1otHXkkn40N28d0Eynp1ik349vmEZaw5GfoNFt4rZAYaROXFWYL4LkIM4XCfIoVvEPhu086-rkEKswS8nhHuoCgmlkgnY0gmlwhKwSV5WKbXVsdGlhZGRyc5YACASsEleVBubIAAoErBJXlQbmyd0DgnJzhQDGAQAAiXNlY3AyNTZrMaEDzmygu0sIMqQX38wEzcf3KKrVZEW-lWPTpqKj-oVgLGSDdGNwgubIg3VkcILmyoV3YWt1Mgk"}'
2026-04-23 05:15:04.038 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 05:15:04.038 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:59079/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.13.210/tcp/52220/p2p/16Uiu2HAmTgzwWbGCvi4thx5qoYYpj3W4zGzDuf5rVDeo8jv1ognU"]'
2026-04-23 05:15:04.041 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:04.041 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:59079/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.107.155/tcp/6982/p2p/16Uiu2HAmEyebw3hhUS8PdSamoEBtH7A72byC9kgaUiJ4Cvr2Gs2M"]'
2026-04-23 05:15:04.076 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:04.076 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:52219/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/0"]'
2026-04-23 05:15:04.080 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:04.080 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6981/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/0"]'
2026-04-23 05:15:04.085 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:04.086 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:59079/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/0"]'
2026-04-23 05:15:04.089 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:04.090 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:59079/lightpush/v1/message" -H "Content-Type: application/json" -d '{"message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1776921304090185807}}'
2026-04-23 05:15:04.093 ERROR [src.node.api_clients.base_client] HTTP error occurred: 503 Server Error: Service Unavailable for url: http://127.0.0.1:59079/lightpush/v1/message. Response content: b'Failed to request a message push: Pubsub topic must be specified when static sharding is enabled'
2026-04-23 05:15:04.166 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-04-23 05:15:04.167 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-04-23 05:15:04.167 DEBUG [src.node.waku_node] Stopping container with id 2474d5a77587
2026-04-23 05:15:04.677 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 05:15:04.677 DEBUG [src.node.waku_node] Stopping container with id e42a8832e4c6
2026-04-23 05:15:05.184 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 05:15:05.185 DEBUG [src.node.waku_node] Stopping container with id cf7da982ff9a
2026-04-23 05:15:05.494 ERROR [src.node.docker_mananger] Max retries reached for container 2474d5a77587. Exiting log stream.
2026-04-23 05:15:05.672 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 05:15:05.675 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-04-23 05:15:05.686 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 05:15:05.705 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 05:15:05.716 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 05:15:05.718 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-04-23 05:15:05.718 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-04-23 05:15:05.720 DEBUG [src.libs.common] Attaching file ./log/docker/lightpush_node1_2026-04-23_05-15-00__5ddd50f8-022e-49e5-9f91-907730b082bd__wakuorg_nwaku:latest.log
2026-04-23 05:15:05.725 DEBUG [src.libs.common] Attaching file ./log/docker/receiving_node1_2026-04-23_05-15-00__5ddd50f8-022e-49e5-9f91-907730b082bd__wakuorg_nwaku:latest.log
2026-04-23 05:15:05.729 DEBUG [src.libs.common] Attaching file ./log/docker/receiving_node2_2026-04-23_05-15-00__5ddd50f8-022e-49e5-9f91-907730b082bd__wakuorg_nwaku:latest.log
2026-04-23 05:15:05.745 DEBUG [tests.conftest] Running fixture setup: test_id
2026-04-23 05:15:05.747 DEBUG [tests.conftest] Running test: test_light_push_with_missing_pubsub_topics with id: 2026-04-23_05-15-05__4a63a4ef-7686-48ca-a0f9-acc2124be0f3
2026-04-23 05:15:05.747 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-04-23 05:15:05.747 DEBUG [src.steps.light_push] Running fixture setup: light_push_setup
2026-04-23 05:15:05.757 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 05:15:05.758 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node1_2026-04-23_05-15-05__4a63a4ef-7686-48ca-a0f9-acc2124be0f3__wakuorg_nwaku:latest.log
2026-04-23 05:15:05.759 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 05:15:05.759 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 05:15:05.761 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 05:15:05.761 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.80.243
2026-04-23 05:15:05.761 DEBUG [src.node.docker_mananger] Generated ports ['53419', '53420', '53421', '53422', '53423']
2026-04-23 05:15:05.761 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 05:15:05.761 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 05:15:05.761 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 05:15:05.762 DEBUG [src.node.docker_mananger] docker run -i -t -p 53419:53419 -p 53420:53420 -p 53421:53421 -p 53422:53422 -p 53423:53423 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=53421 --rest-port=53419 --tcp-port=53420 --discv5-udp-port=53422 --rest-address=0.0.0.0 --nat=extip:172.18.80.243 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=e11da3d452c64f270a0aaeb08cabdf97780cacfae177e78fafe750ebbe44cd7f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=53423 --metrics-logging=true --lightpush=true --relay=true
2026-04-23 05:15:05.953 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.80.243 waku 8f43cbf44182bc657d1c4cb607839bf1138f98fd086cfdb219222ee10959a71a
2026-04-23 05:15:05.989 DEBUG [src.node.docker_mananger] Container started with ID 8f43cbf44182. Setting up logs at ./log/docker/receiving_node1_2026-04-23_05-15-05__4a63a4ef-7686-48ca-a0f9-acc2124be0f3__wakuorg_nwaku:latest.log
2026-04-23 05:15:05.990 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 53419
2026-04-23 05:15:05.990 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 05:15:06.032 ERROR [src.node.docker_mananger] Max retries reached for container e42a8832e4c6. Exiting log stream.
2026-04-23 05:15:06.572 ERROR [src.node.docker_mananger] Max retries reached for container cf7da982ff9a. Exiting log stream.
2026-04-23 05:15:06.990 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53419/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:06.993 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 05:15:06.994 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 05:15:06.994 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53419/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:06.996 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.80.243/tcp/53420/p2p/16Uiu2HAmMTWxeeDg8KtkspQuTdTt2LJ9iuXgpt3W2aT6M3ESQnM3","/ip4/172.18.80.243/tcp/53421/ws/p2p/16Uiu2HAmMTWxeeDg8KtkspQuTdTt2LJ9iuXgpt3W2aT6M3ESQnM3"],"enrUri":"enr:-L24QA_Ar2JDVzCKkDuutj5Lwa6oZpetUI0RstKE_Tgy8c7wAo2m5v7uusY9UtYG5a2NfHCjffqFKpceNh2CMacyWMYCgmlkgnY0gmlwhKwSUPOKbXVsdGlhZGRyc5YACASsElDzBtCsAAoErBJQ8wbQrd0DgnJzhQDGAQAAiXNlY3AyNTZrMaEDgsjStQHWLia3-SEhI9RTOCpfqN99448PH0oRWuGDMKaDdGNwgtCsg3VkcILQroV3YWt1Mgk"}'
2026-04-23 05:15:06.996 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 05:15:07.004 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 05:15:07.005 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node2_2026-04-23_05-15-05__4a63a4ef-7686-48ca-a0f9-acc2124be0f3__wakuorg_nwaku:latest.log
2026-04-23 05:15:07.005 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 05:15:07.005 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 05:15:07.007 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 05:15:07.007 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.187.115
2026-04-23 05:15:07.007 DEBUG [src.node.docker_mananger] Generated ports ['24151', '24152', '24153', '24154', '24155']
2026-04-23 05:15:07.007 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 05:15:07.007 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 05:15:07.007 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 05:15:07.007 DEBUG [src.node.docker_mananger] docker run -i -t -p 24151:24151 -p 24152:24152 -p 24153:24153 -p 24154:24154 -p 24155:24155 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=24153 --rest-port=24151 --tcp-port=24152 --discv5-udp-port=24154 --rest-address=0.0.0.0 --nat=extip:172.18.187.115 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=b2f30aa9f8a409a0ed2bff7acccdbfa12aeaaf4dbf4951bf4bd4d5abac34aad7 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=24155 --metrics-logging=true --lightpush=false --relay=true
2026-04-23 05:15:07.211 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.187.115 waku c39e74f246a3612cf705657b4ea9efcb1b256b53b3702eed05cd89b3b52892b0
2026-04-23 05:15:07.243 DEBUG [src.node.docker_mananger] Container started with ID c39e74f246a3. Setting up logs at ./log/docker/receiving_node2_2026-04-23_05-15-05__4a63a4ef-7686-48ca-a0f9-acc2124be0f3__wakuorg_nwaku:latest.log
2026-04-23 05:15:07.244 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 24151
2026-04-23 05:15:07.245 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 05:15:08.245 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24151/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:08.249 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 05:15:08.249 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 05:15:08.249 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24151/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:08.252 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.187.115/tcp/24152/p2p/16Uiu2HAm81Yvnmg28EYJp2D37eZBJARZyzfSqzZDqsJs66Lw9ZE3","/ip4/172.18.187.115/tcp/24153/ws/p2p/16Uiu2HAm81Yvnmg28EYJp2D37eZBJARZyzfSqzZDqsJs66Lw9ZE3"],"enrUri":"enr:-L24QHBxHMrAshx3SWihI28Bpw66fxK4D0xLyNRdbRGj6u3UT8tjHahOCiY3Ndbbh2gSZXf8RNh_Qc8xVIce0dT1MrkCgmlkgnY0gmlwhKwSu3OKbXVsdGlhZGRyc5YACASsErtzBl5YAAoErBK7cwZeWd0DgnJzhQDGAQAAiXNlY3AyNTZrMaECuvoEY6W04P1-HKsNB4ByWhZ9H5Pa_DmeiNSqfprYHLSDdGNwgl5Yg3VkcIJeWoV3YWt1MgE"}'
2026-04-23 05:15:08.252 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 05:15:08.252 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:24151/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.80.243/tcp/53420/p2p/16Uiu2HAmMTWxeeDg8KtkspQuTdTt2LJ9iuXgpt3W2aT6M3ESQnM3"]'
2026-04-23 05:15:08.285 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:08.296 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 05:15:08.297 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node1_2026-04-23_05-15-05__4a63a4ef-7686-48ca-a0f9-acc2124be0f3__wakuorg_nwaku:latest.log
2026-04-23 05:15:08.298 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 05:15:08.298 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 05:15:08.301 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 05:15:08.301 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.110.100
2026-04-23 05:15:08.302 DEBUG [src.node.docker_mananger] Generated ports ['60031', '60032', '60033', '60034', '60035']
2026-04-23 05:15:08.302 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 05:15:08.302 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 05:15:08.302 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 05:15:08.302 DEBUG [src.node.docker_mananger] docker run -i -t -p 60031:60031 -p 60032:60032 -p 60033:60033 -p 60034:60034 -p 60035:60035 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=60033 --rest-port=60031 --tcp-port=60032 --discv5-udp-port=60034 --rest-address=0.0.0.0 --nat=extip:172.18.110.100 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=bccb8fe550dfa06fdbeb4d2eab8ab365ceded1fb1fad4a9de1dcb9e504f1ba69 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=60035 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QA_Ar2JDVzCKkDuutj5Lwa6oZpetUI0RstKE_Tgy8c7wAo2m5v7uusY9UtYG5a2NfHCjffqFKpceNh2CMacyWMYCgmlkgnY0gmlwhKwSUPOKbXVsdGlhZGRyc5YACASsElDzBtCsAAoErBJQ8wbQrd0DgnJzhQDGAQAAiXNlY3AyNTZrMaEDgsjStQHWLia3-SEhI9RTOCpfqN99448PH0oRWuGDMKaDdGNwgtCsg3VkcILQroV3YWt1Mgk --lightpushnode=/ip4/172.18.80.243/tcp/53420/p2p/16Uiu2HAmMTWxeeDg8KtkspQuTdTt2LJ9iuXgpt3W2aT6M3ESQnM3 --lightpush=true --relay=true
2026-04-23 05:15:08.496 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.110.100 waku 8ebe3982dd946a4728c7ee2cab6905108f220ce894e0fd01f1b0034c798a142b
2026-04-23 05:15:08.530 DEBUG [src.node.docker_mananger] Container started with ID 8ebe3982dd94. Setting up logs at ./log/docker/lightpush_node1_2026-04-23_05-15-05__4a63a4ef-7686-48ca-a0f9-acc2124be0f3__wakuorg_nwaku:latest.log
2026-04-23 05:15:08.530 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 60031
2026-04-23 05:15:08.530 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 05:15:09.531 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60031/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:09.536 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 05:15:09.536 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 05:15:09.537 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60031/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:09.541 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.110.100/tcp/60032/p2p/16Uiu2HAmCYqLoz2BgLXdv5jwPgKpuBacUpkPk68ZySx5dgDFt7Nu","/ip4/172.18.110.100/tcp/60033/ws/p2p/16Uiu2HAmCYqLoz2BgLXdv5jwPgKpuBacUpkPk68ZySx5dgDFt7Nu"],"enrUri":"enr:-L24QDn8WDaYGMs2prBhxtjXr10vBswNYpbmXSmoJnPKCqBLai07nuAi-wEhXzus1ZQ3sg5FpBb6DAZFKeSwCwIUUlgCgmlkgnY0gmlwhKwSbmSKbXVsdGlhZGRyc5YACASsEm5kBuqAAAoErBJuZAbqgd0DgnJzhQDGAQAAiXNlY3AyNTZrMaEC_mxUWHgyYDUf8HyH-PvAxoMVRIjcjUf0uJVK6ug8j-aDdGNwguqAg3VkcILqgoV3YWt1Mgk"}'
2026-04-23 05:15:09.542 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 05:15:09.542 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60031/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.80.243/tcp/53420/p2p/16Uiu2HAmMTWxeeDg8KtkspQuTdTt2LJ9iuXgpt3W2aT6M3ESQnM3"]'
2026-04-23 05:15:09.588 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:09.589 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60031/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.187.115/tcp/24152/p2p/16Uiu2HAm81Yvnmg28EYJp2D37eZBJARZyzfSqzZDqsJs66Lw9ZE3"]'
2026-04-23 05:15:09.623 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:09.625 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:53419/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/0"]'
2026-04-23 05:15:09.633 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:09.635 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:24151/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/0"]'
2026-04-23 05:15:09.644 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:09.644 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60031/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/0"]'
2026-04-23 05:15:09.648 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:09.651 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60031/lightpush/v1/message" -H "Content-Type: application/json" -d '{"message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1776921309651472230}}'
2026-04-23 05:15:09.659 ERROR [src.node.api_clients.base_client] HTTP error occurred: 503 Server Error: Service Unavailable for url: http://127.0.0.1:60031/lightpush/v1/message. Response content: b'Failed to request a message push: Pubsub topic must be specified when static sharding is enabled'
2026-04-23 05:15:09.715 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-04-23 05:15:09.718 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-04-23 05:15:09.719 DEBUG [src.node.waku_node] Stopping container with id 8f43cbf44182
2026-04-23 05:15:10.194 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 05:15:10.195 DEBUG [src.node.waku_node] Stopping container with id c39e74f246a3
2026-04-23 05:15:10.768 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 05:15:10.768 DEBUG [src.node.waku_node] Stopping container with id 8ebe3982dd94
2026-04-23 05:15:11.010 ERROR [src.node.docker_mananger] Max retries reached for container 8f43cbf44182. Exiting log stream.
2026-04-23 05:15:11.220 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 05:15:11.222 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-04-23 05:15:11.236 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 05:15:11.245 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 05:15:11.255 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 05:15:11.256 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-04-23 05:15:11.256 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-04-23 05:15:11.256 DEBUG [src.libs.common] Attaching file ./log/docker/lightpush_node1_2026-04-23_05-15-05__4a63a4ef-7686-48ca-a0f9-acc2124be0f3__wakuorg_nwaku:latest.log
2026-04-23 05:15:11.257 DEBUG [src.libs.common] Attaching file ./log/docker/receiving_node2_2026-04-23_05-15-05__4a63a4ef-7686-48ca-a0f9-acc2124be0f3__wakuorg_nwaku:latest.log
2026-04-23 05:15:11.257 DEBUG [src.libs.common] Attaching file ./log/docker/receiving_node1_2026-04-23_05-15-05__4a63a4ef-7686-48ca-a0f9-acc2124be0f3__wakuorg_nwaku:latest.log
2026-04-23 05:15:11.261 DEBUG [tests.conftest] Running fixture setup: test_id
2026-04-23 05:15:11.261 DEBUG [tests.conftest] Running test: test_light_push_with_missing_pubsub_topics with id: 2026-04-23_05-15-11__bdee35a9-d9b5-4924-b72c-c6e53808eac9
2026-04-23 05:15:11.261 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-04-23 05:15:11.262 DEBUG [src.steps.light_push] Running fixture setup: light_push_setup
2026-04-23 05:15:11.269 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 05:15:11.270 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node1_2026-04-23_05-15-11__bdee35a9-d9b5-4924-b72c-c6e53808eac9__wakuorg_nwaku:latest.log
2026-04-23 05:15:11.270 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 05:15:11.270 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 05:15:11.271 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 05:15:11.271 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.43.139
2026-04-23 05:15:11.272 DEBUG [src.node.docker_mananger] Generated ports ['33954', '33955', '33956', '33957', '33958']
2026-04-23 05:15:11.272 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 05:15:11.272 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 05:15:11.272 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 05:15:11.272 DEBUG [src.node.docker_mananger] docker run -i -t -p 33954:33954 -p 33955:33955 -p 33956:33956 -p 33957:33957 -p 33958:33958 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=33956 --rest-port=33954 --tcp-port=33955 --discv5-udp-port=33957 --rest-address=0.0.0.0 --nat=extip:172.18.43.139 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=dc00cc5574f9dad05bcc195ea19fe759bb7577807fb26fa30c1af2a7b3efcd8c --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=33958 --metrics-logging=true --lightpush=true --relay=true
2026-04-23 05:15:11.473 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.43.139 waku 0e6edf71046425a5d415272d4a4d0a8157ae273bf920a9918916396c2d91cd92
2026-04-23 05:15:11.508 DEBUG [src.node.docker_mananger] Container started with ID 0e6edf710464. Setting up logs at ./log/docker/receiving_node1_2026-04-23_05-15-11__bdee35a9-d9b5-4924-b72c-c6e53808eac9__wakuorg_nwaku:latest.log
2026-04-23 05:15:11.508 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 33954
2026-04-23 05:15:11.508 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 05:15:11.583 ERROR [src.node.docker_mananger] Max retries reached for container c39e74f246a3. Exiting log stream.
2026-04-23 05:15:12.048 ERROR [src.node.docker_mananger] Max retries reached for container 8ebe3982dd94. Exiting log stream.
2026-04-23 05:15:12.509 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33954/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:12.512 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 05:15:12.512 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 05:15:12.512 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33954/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:12.515 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.43.139/tcp/33955/p2p/16Uiu2HAm9RPxBq4SUqZsF3JLn3zkxZ2fqyKLb98MmWhpfaHQrRud","/ip4/172.18.43.139/tcp/33956/ws/p2p/16Uiu2HAm9RPxBq4SUqZsF3JLn3zkxZ2fqyKLb98MmWhpfaHQrRud"],"enrUri":"enr:-L24QLZEDL0UMmeliWJiN5neY5t-_YtiMzu3s4uuuoHUuIdPY7uQoBoT8msJcb91-B66X5iftgDaHDrTnTS2PnQSu-gCgmlkgnY0gmlwhKwSK4uKbXVsdGlhZGRyc5YACASsEiuLBoSjAAoErBIriwaEpN0DgnJzhQDGAQAAiXNlY3AyNTZrMaECz_F-kSjH0hRiciHnIfo6Lj44C3HwVjbQX_naGbQf_6SDdGNwgoSjg3VkcIKEpYV3YWt1Mgk"}'
2026-04-23 05:15:12.515 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 05:15:12.524 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 05:15:12.524 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node2_2026-04-23_05-15-11__bdee35a9-d9b5-4924-b72c-c6e53808eac9__wakuorg_nwaku:latest.log
2026-04-23 05:15:12.524 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 05:15:12.524 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 05:15:12.526 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 05:15:12.526 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.66.56
2026-04-23 05:15:12.526 DEBUG [src.node.docker_mananger] Generated ports ['4140', '4141', '4142', '4143', '4144']
2026-04-23 05:15:12.526 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 05:15:12.526 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 05:15:12.526 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 05:15:12.527 DEBUG [src.node.docker_mananger] docker run -i -t -p 4140:4140 -p 4141:4141 -p 4142:4142 -p 4143:4143 -p 4144:4144 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=4142 --rest-port=4140 --tcp-port=4141 --discv5-udp-port=4143 --rest-address=0.0.0.0 --nat=extip:172.18.66.56 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=b28fbadfa0e6ae89f5a048f6ab73afd21d85371ba4e3ca2fc0a0512bc02123cd --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=4144 --metrics-logging=true --lightpush=false --relay=true
2026-04-23 05:15:12.718 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.66.56 waku 74c089f1225cc5e54dcde010c8c7a0c19a0e7d375edc55ffd9b1e1e6b53c66bd
2026-04-23 05:15:12.759 DEBUG [src.node.docker_mananger] Container started with ID 74c089f1225c. Setting up logs at ./log/docker/receiving_node2_2026-04-23_05-15-11__bdee35a9-d9b5-4924-b72c-c6e53808eac9__wakuorg_nwaku:latest.log
2026-04-23 05:15:12.759 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 4140
2026-04-23 05:15:12.760 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 05:15:13.760 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4140/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:13.763 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 05:15:13.764 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 05:15:13.764 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4140/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:13.767 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.66.56/tcp/4141/p2p/16Uiu2HAmRtYVP5hTMEoMaKVQ2HVSS7pe1ZYM9LGC8R8RHDLFd2By","/ip4/172.18.66.56/tcp/4142/ws/p2p/16Uiu2HAmRtYVP5hTMEoMaKVQ2HVSS7pe1ZYM9LGC8R8RHDLFd2By"],"enrUri":"enr:-L24QEFXn9WMGkaQ62SZwzdctjSrSGnqNjw3wpXwXC8XfNHXQMRzwAxuTHdaPA7udhacv-BP02KYarkHNIbMGh3i4l8CgmlkgnY0gmlwhKwSQjiKbXVsdGlhZGRyc5YACASsEkI4BhAtAAoErBJCOAYQLt0DgnJzhQDGAQAAiXNlY3AyNTZrMaEDxKDSIV6PPZAKWFg89-m_1zgA_BVUo70vheWRbwFc2ACDdGNwghAtg3VkcIIQL4V3YWt1MgE"}'
2026-04-23 05:15:13.767 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 05:15:13.767 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:4140/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.43.139/tcp/33955/p2p/16Uiu2HAm9RPxBq4SUqZsF3JLn3zkxZ2fqyKLb98MmWhpfaHQrRud"]'
2026-04-23 05:15:13.797 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:13.815 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 05:15:13.815 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node1_2026-04-23_05-15-11__bdee35a9-d9b5-4924-b72c-c6e53808eac9__wakuorg_nwaku:latest.log
2026-04-23 05:15:13.816 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 05:15:13.816 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 05:15:13.818 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 05:15:13.818 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.116.213
2026-04-23 05:15:13.819 DEBUG [src.node.docker_mananger] Generated ports ['19636', '19637', '19638', '19639', '19640']
2026-04-23 05:15:13.819 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 05:15:13.819 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 05:15:13.819 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 05:15:13.820 DEBUG [src.node.docker_mananger] docker run -i -t -p 19636:19636 -p 19637:19637 -p 19638:19638 -p 19639:19639 -p 19640:19640 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=19638 --rest-port=19636 --tcp-port=19637 --discv5-udp-port=19639 --rest-address=0.0.0.0 --nat=extip:172.18.116.213 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=dbcda68e7528d5f7ac7dadcc8b7f0e89c4aa435314619d14e12ffcd8e1b8fe34 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=19640 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QLZEDL0UMmeliWJiN5neY5t-_YtiMzu3s4uuuoHUuIdPY7uQoBoT8msJcb91-B66X5iftgDaHDrTnTS2PnQSu-gCgmlkgnY0gmlwhKwSK4uKbXVsdGlhZGRyc5YACASsEiuLBoSjAAoErBIriwaEpN0DgnJzhQDGAQAAiXNlY3AyNTZrMaECz_F-kSjH0hRiciHnIfo6Lj44C3HwVjbQX_naGbQf_6SDdGNwgoSjg3VkcIKEpYV3YWt1Mgk --lightpushnode=/ip4/172.18.43.139/tcp/33955/p2p/16Uiu2HAm9RPxBq4SUqZsF3JLn3zkxZ2fqyKLb98MmWhpfaHQrRud --lightpush=true --relay=true
2026-04-23 05:15:14.021 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.116.213 waku 6d1e3dd54ec32a03ed602dac3690e0ae30e6fdd781ac7bebd35070616b15f145
2026-04-23 05:15:14.059 DEBUG [src.node.docker_mananger] Container started with ID 6d1e3dd54ec3. Setting up logs at ./log/docker/lightpush_node1_2026-04-23_05-15-11__bdee35a9-d9b5-4924-b72c-c6e53808eac9__wakuorg_nwaku:latest.log
2026-04-23 05:15:14.060 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 19636
2026-04-23 05:15:14.060 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 05:15:15.061 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19636/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:15.065 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 05:15:15.065 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 05:15:15.065 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19636/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 05:15:15.068 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.116.213/tcp/19637/p2p/16Uiu2HAmS2GiozRC8kKWKVovdx86cndocPURYNU8YrcaWXnZpsXS","/ip4/172.18.116.213/tcp/19638/ws/p2p/16Uiu2HAmS2GiozRC8kKWKVovdx86cndocPURYNU8YrcaWXnZpsXS"],"enrUri":"enr:-L24QLnembRhG167xyZYgt1bgc6Ks10ha2Ts-G7C9XQoRANnbbZVY7FxMMCT7nZi2z2_lqmb12tVq4J-1u-eKX1QNz0CgmlkgnY0gmlwhKwSdNWKbXVsdGlhZGRyc5YACASsEnTVBky1AAoErBJ01QZMtt0DgnJzhQDGAQAAiXNlY3AyNTZrMaEDxpujSaYBl9hXK3fOQ3QDDks6KxzIQXAW8uoi-uT8G2WDdGNwgky1g3VkcIJMt4V3YWt1Mgk"}'
2026-04-23 05:15:15.068 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 05:15:15.068 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19636/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.43.139/tcp/33955/p2p/16Uiu2HAm9RPxBq4SUqZsF3JLn3zkxZ2fqyKLb98MmWhpfaHQrRud"]'
2026-04-23 05:15:15.103 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:15.104 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19636/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.66.56/tcp/4141/p2p/16Uiu2HAmRtYVP5hTMEoMaKVQ2HVSS7pe1ZYM9LGC8R8RHDLFd2By"]'
2026-04-23 05:15:15.141 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:15.147 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:33954/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/0"]'
2026-04-23 05:15:15.152 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:15.152 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:4140/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/0"]'
2026-04-23 05:15:15.155 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:15.155 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19636/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/0"]'
2026-04-23 05:15:15.157 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 05:15:15.158 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19636/lightpush/v1/message" -H "Content-Type: application/json" -d '{"message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1776921315158793299}}'
2026-04-23 05:15:15.162 ERROR [src.node.api_clients.base_client] HTTP error occurred: 503 Server Error: Service Unavailable for url: http://127.0.0.1:19636/lightpush/v1/message. Response content: b'Failed to request a message push: Pubsub topic must be specified when static sharding is enabled'
2026-04-23 05:15:15.201 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-04-23 05:15:15.202 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-04-23 05:15:15.203 DEBUG [src.node.waku_node] Stopping container with id 0e6edf710464
2026-04-23 05:15:15.695 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 05:15:15.697 DEBUG [src.node.waku_node] Stopping container with id 74c089f1225c
2026-04-23 05:15:16.189 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 05:15:16.190 DEBUG [src.node.waku_node] Stopping container with id 6d1e3dd54ec3
2026-04-23 05:15:16.556 ERROR [src.node.docker_mananger] Max retries reached for container 0e6edf710464. Exiting log stream.
2026-04-23 05:15:16.630 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 05:15:16.633 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-04-23 05:15:16.653 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 05:15:16.663 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 05:15:16.673 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 05:15:16.673 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-04-23 05:15:16.674 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-04-23 05:15:16.674 DEBUG [src.libs.common] Attaching file ./log/docker/receiving_node2_2026-04-23_05-15-11__bdee35a9-d9b5-4924-b72c-c6e53808eac9__wakuorg_nwaku:latest.log
2026-04-23 05:15:16.675 DEBUG [src.libs.common] Attaching file ./log/docker/receiving_node1_2026-04-23_05-15-11__bdee35a9-d9b5-4924-b72c-c6e53808eac9__wakuorg_nwaku:latest.log
2026-04-23 05:15:16.675 DEBUG [src.libs.common] Attaching file ./log/docker/lightpush_node1_2026-04-23_05-15-11__bdee35a9-d9b5-4924-b72c-c6e53808eac9__wakuorg_nwaku:latest.log