97 lines
18 KiB
Plaintext

2026-03-13 04:34:54.070 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-13 04:34:54.070 DEBUG [tests.conftest] Running test: test_light_push_while_peer_is_paused with id: 2026-03-13_04-34-54__59cd1026-a768-4280-941f-5d8ee09d0a81
2026-03-13 04:34:54.070 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-13 04:34:54.071 DEBUG [src.steps.light_push] Running fixture setup: light_push_setup
2026-03-13 04:34:54.078 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-13 04:34:54.078 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node1_2026-03-13_04-34-54__59cd1026-a768-4280-941f-5d8ee09d0a81__wakuorg_nwaku:latest.log
2026-03-13 04:34:54.078 DEBUG [src.node.waku_node] Starting Node...
2026-03-13 04:34:54.078 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-13 04:34:54.080 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-13 04:34:54.080 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.223.54
2026-03-13 04:34:54.080 DEBUG [src.node.docker_mananger] Generated ports ['16128', '16129', '16130', '16131', '16132']
2026-03-13 04:34:54.080 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-13 04:34:54.080 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-13 04:34:54.080 DEBUG [src.node.waku_node] Using volumes []
2026-03-13 04:34:54.081 DEBUG [src.node.docker_mananger] docker run -i -t -p 16128:16128 -p 16129:16129 -p 16130:16130 -p 16131:16131 -p 16132:16132 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=16130 --rest-port=16128 --tcp-port=16129 --discv5-udp-port=16131 --rest-address=0.0.0.0 --nat=extip:172.18.223.54 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=01846a2fa9635465b4daafa037d60f60fb3ef87dca6ae0c6370567c9da38c22a --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=16132 --metrics-logging=true --lightpush=true --relay=true
2026-03-13 04:34:54.271 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.223.54 waku eb330c1a8185baa0c7d0070ee98af6c573e9683444e312e9de26daddcea4dffd
2026-03-13 04:34:54.307 DEBUG [src.node.docker_mananger] Container started with ID eb330c1a8185. Setting up logs at ./log/docker/receiving_node1_2026-03-13_04-34-54__59cd1026-a768-4280-941f-5d8ee09d0a81__wakuorg_nwaku:latest.log
2026-03-13 04:34:54.307 ERROR [src.node.docker_mananger] Max retries reached for container 3d9bbae40d36. Exiting log stream.
2026-03-13 04:34:54.309 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 16128
2026-03-13 04:34:54.311 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-13 04:34:54.872 ERROR [src.node.docker_mananger] Max retries reached for container 6870f9fee455. Exiting log stream.
2026-03-13 04:34:55.311 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16128/health" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:34:55.314 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-13 04:34:55.315 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-13 04:34:55.315 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16128/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:34:55.317 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.223.54/tcp/16129/p2p/16Uiu2HAm4CshcSfj3namYARq3tqEuCrJ8s9XGgqEkf6ux3mxgocn","/ip4/172.18.223.54/tcp/16130/ws/p2p/16Uiu2HAm4CshcSfj3namYARq3tqEuCrJ8s9XGgqEkf6ux3mxgocn"],"enrUri":"enr:-L24QDSSQUEn4mY28Nx8Fc-A0GK7FBAw-qedSZgmsP_dwERwQ0_YKJwEBopt60Zyh5hCW3HQ5907QoHm9r5y34V2GW0CgmlkgnY0gmlwhKwS3zaKbXVsdGlhZGRyc5YACASsEt82Bj8BAAoErBLfNgY_At0DgnJzhQADAQAAiXNlY3AyNTZrMaECgnHignugVv08hAOMxjvo2RKzPxZ_Miwu9bq7TEx4CpuDdGNwgj8Bg3VkcII_A4V3YWt1Mgk"}'
2026-03-13 04:34:55.318 INFO [src.node.waku_node] REST service is ready !!
2026-03-13 04:34:55.325 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-13 04:34:55.325 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node2_2026-03-13_04-34-54__59cd1026-a768-4280-941f-5d8ee09d0a81__wakuorg_nwaku:latest.log
2026-03-13 04:34:55.325 DEBUG [src.node.waku_node] Starting Node...
2026-03-13 04:34:55.325 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-13 04:34:55.327 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-13 04:34:55.327 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.139.26
2026-03-13 04:34:55.327 DEBUG [src.node.docker_mananger] Generated ports ['38559', '38560', '38561', '38562', '38563']
2026-03-13 04:34:55.327 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-13 04:34:55.327 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-13 04:34:55.328 DEBUG [src.node.waku_node] Using volumes []
2026-03-13 04:34:55.328 DEBUG [src.node.docker_mananger] docker run -i -t -p 38559:38559 -p 38560:38560 -p 38561:38561 -p 38562:38562 -p 38563:38563 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=38561 --rest-port=38559 --tcp-port=38560 --discv5-udp-port=38562 --rest-address=0.0.0.0 --nat=extip:172.18.139.26 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=528fc936bd3a51ac4da53fe4bca73fea6a6fbf6cd4defc8a440ea4bf1ecaa6ba --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=38563 --metrics-logging=true --lightpush=false --relay=true
2026-03-13 04:34:55.516 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.139.26 waku 13f7d6c188998a9d30cc23c90605612adb5f336b830a1e090a30d8b31137f185
2026-03-13 04:34:55.552 DEBUG [src.node.docker_mananger] Container started with ID 13f7d6c18899. Setting up logs at ./log/docker/receiving_node2_2026-03-13_04-34-54__59cd1026-a768-4280-941f-5d8ee09d0a81__wakuorg_nwaku:latest.log
2026-03-13 04:34:55.553 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 38559
2026-03-13 04:34:55.554 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-13 04:34:56.554 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38559/health" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:34:56.557 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-13 04:34:56.558 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-13 04:34:56.558 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38559/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:34:56.560 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.139.26/tcp/38560/p2p/16Uiu2HAmLZvpFeuGfEKpZk3JQNU3NjXEWwQ9NvZdnHHTkpgfLxeT","/ip4/172.18.139.26/tcp/38561/ws/p2p/16Uiu2HAmLZvpFeuGfEKpZk3JQNU3NjXEWwQ9NvZdnHHTkpgfLxeT"],"enrUri":"enr:-L24QCpqIJoxEWVAftQ_8O4KD5lkcrZg7ZzlYW-henOFqgkLVqXm459Rkw1GPoiXEzjo-6XzjdYxLudUNGa5pg13RH4CgmlkgnY0gmlwhKwSixqKbXVsdGlhZGRyc5YACASsEosaBpagAAoErBKLGgaWod0DgnJzhQADAQAAiXNlY3AyNTZrMaEDdZGYHbLmd66OWqDQFBjbztv5rw_dMdydQrnGkuaQMzCDdGNwgpagg3VkcIKWooV3YWt1MgE"}'
2026-03-13 04:34:56.560 INFO [src.node.waku_node] REST service is ready !!
2026-03-13 04:34:56.561 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38559/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.223.54/tcp/16129/p2p/16Uiu2HAm4CshcSfj3namYARq3tqEuCrJ8s9XGgqEkf6ux3mxgocn"]'
2026-03-13 04:34:56.602 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:34:56.611 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-13 04:34:56.611 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node1_2026-03-13_04-34-54__59cd1026-a768-4280-941f-5d8ee09d0a81__wakuorg_nwaku:latest.log
2026-03-13 04:34:56.612 DEBUG [src.node.waku_node] Starting Node...
2026-03-13 04:34:56.612 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-13 04:34:56.613 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-13 04:34:56.613 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.38.152
2026-03-13 04:34:56.614 DEBUG [src.node.docker_mananger] Generated ports ['37436', '37437', '37438', '37439', '37440']
2026-03-13 04:34:56.614 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-13 04:34:56.614 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-13 04:34:56.614 DEBUG [src.node.waku_node] Using volumes []
2026-03-13 04:34:56.614 DEBUG [src.node.docker_mananger] docker run -i -t -p 37436:37436 -p 37437:37437 -p 37438:37438 -p 37439:37439 -p 37440:37440 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=37438 --rest-port=37436 --tcp-port=37437 --discv5-udp-port=37439 --rest-address=0.0.0.0 --nat=extip:172.18.38.152 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=aced3b3ad4a02608e38eff34b1fdc9f726ab3cff06d98acb7ffceedb2cfbb182 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=37440 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QDSSQUEn4mY28Nx8Fc-A0GK7FBAw-qedSZgmsP_dwERwQ0_YKJwEBopt60Zyh5hCW3HQ5907QoHm9r5y34V2GW0CgmlkgnY0gmlwhKwS3zaKbXVsdGlhZGRyc5YACASsEt82Bj8BAAoErBLfNgY_At0DgnJzhQADAQAAiXNlY3AyNTZrMaECgnHignugVv08hAOMxjvo2RKzPxZ_Miwu9bq7TEx4CpuDdGNwgj8Bg3VkcII_A4V3YWt1Mgk --lightpushnode=/ip4/172.18.223.54/tcp/16129/p2p/16Uiu2HAm4CshcSfj3namYARq3tqEuCrJ8s9XGgqEkf6ux3mxgocn --lightpush=true --relay=true
2026-03-13 04:34:56.808 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.38.152 waku 72307c0d1d08c535777d1e126db5b4cb187dc4112ee9236426aba695bb05f9f8
2026-03-13 04:34:56.845 DEBUG [src.node.docker_mananger] Container started with ID 72307c0d1d08. Setting up logs at ./log/docker/lightpush_node1_2026-03-13_04-34-54__59cd1026-a768-4280-941f-5d8ee09d0a81__wakuorg_nwaku:latest.log
2026-03-13 04:34:56.846 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 37436
2026-03-13 04:34:56.846 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-13 04:34:57.846 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:37436/health" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:34:57.849 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-13 04:34:57.849 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-13 04:34:57.850 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:37436/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:34:57.852 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.38.152/tcp/37437/p2p/16Uiu2HAmEhZxpCVnYxoHidkwYeXmEwWjE2e1SpWjSjy31FwgxX9v","/ip4/172.18.38.152/tcp/37438/ws/p2p/16Uiu2HAmEhZxpCVnYxoHidkwYeXmEwWjE2e1SpWjSjy31FwgxX9v"],"enrUri":"enr:-L24QC8mNIUQk3rqo0FqCghF6u5TwKL-t6nCJ2GuiWPMp6BzHwntz40ma99L56Vl5f8VoWA9gOK3BhedeGnUl0AKAAcCgmlkgnY0gmlwhKwSJpiKbXVsdGlhZGRyc5YACASsEiaYBpI9AAoErBImmAaSPt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDHmCKjrPrnhiMt2tuQ_3pHlZyz2Vl90uKa7Hr2nMo0YWDdGNwgpI9g3VkcIKSP4V3YWt1Mgk"}'
2026-03-13 04:34:57.852 INFO [src.node.waku_node] REST service is ready !!
2026-03-13 04:34:57.852 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:37436/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.223.54/tcp/16129/p2p/16Uiu2HAm4CshcSfj3namYARq3tqEuCrJ8s9XGgqEkf6ux3mxgocn"]'
2026-03-13 04:34:57.896 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:34:57.898 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:37436/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.139.26/tcp/38560/p2p/16Uiu2HAmLZvpFeuGfEKpZk3JQNU3NjXEWwQ9NvZdnHHTkpgfLxeT"]'
2026-03-13 04:34:57.928 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:34:57.930 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16128/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-13 04:34:57.946 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:34:57.947 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38559/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-13 04:34:57.952 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:34:57.955 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:37436/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-13 04:34:57.960 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:34:57.962 DEBUG [src.node.waku_node] Stopping container with id eb330c1a8185
2026-03-13 04:34:58.563 DEBUG [src.node.waku_node] Container stopped.
2026-03-13 04:34:58.565 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:37436/lightpush/v1/message" -H "Content-Type: application/json" -d '{"pubsubTopic": "/waku/2/rs/3/0", "message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1773376497962907321}}'
2026-03-13 04:34:59.377 ERROR [src.node.docker_mananger] Max retries reached for container eb330c1a8185. Exiting log stream.
2026-03-13 04:35:03.574 ERROR [src.node.api_clients.base_client] HTTP error occurred: 503 Server Error: Service Unavailable for url: http://127.0.0.1:37436/lightpush/v1/message. Response content: b'Failed to request a message push: dial_failure'
2026-03-13 04:35:03.576 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-13 04:35:03.577 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-13 04:35:03.577 DEBUG [src.node.waku_node] Stopping container with id 13f7d6c18899
2026-03-13 04:35:04.156 DEBUG [src.node.waku_node] Container stopped.
2026-03-13 04:35:04.157 DEBUG [src.node.waku_node] Stopping container with id 72307c0d1d08
2026-03-13 04:35:04.695 DEBUG [src.node.waku_node] Container stopped.
2026-03-13 04:35:04.697 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-13 04:35:04.705 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-13 04:35:04.713 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-13 04:35:04.720 DEBUG [src.node.docker_mananger] No errors found in the waku logs.