97 lines
18 KiB
Plaintext

2026-02-26 04:36:24.733 DEBUG [tests.conftest] Running fixture setup: test_id
2026-02-26 04:36:24.733 DEBUG [tests.conftest] Running test: test_light_push_while_peer_is_paused with id: 2026-02-26_04-36-24__f75efd15-da5d-4f2c-8b63-c01846825c67
2026-02-26 04:36:24.733 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-02-26 04:36:24.734 DEBUG [src.steps.light_push] Running fixture setup: light_push_setup
2026-02-26 04:36:24.740 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-26 04:36:24.741 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node1_2026-02-26_04-36-24__f75efd15-da5d-4f2c-8b63-c01846825c67__wakuorg_nwaku:latest.log
2026-02-26 04:36:24.741 DEBUG [src.node.waku_node] Starting Node...
2026-02-26 04:36:24.741 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-26 04:36:24.742 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-26 04:36:24.742 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.40.15
2026-02-26 04:36:24.743 DEBUG [src.node.docker_mananger] Generated ports ['4358', '4359', '4360', '4361', '4362']
2026-02-26 04:36:24.743 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-26 04:36:24.743 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-26 04:36:24.743 DEBUG [src.node.waku_node] Using volumes []
2026-02-26 04:36:24.743 DEBUG [src.node.docker_mananger] docker run -i -t -p 4358:4358 -p 4359:4359 -p 4360:4360 -p 4361:4361 -p 4362:4362 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=4360 --rest-port=4358 --tcp-port=4359 --discv5-udp-port=4361 --rest-address=0.0.0.0 --nat=extip:172.18.40.15 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=8da1ff996e932b0ef0caba3d7dc2ee1b2b3efcaa51eae38e60eb05f6eb936df2 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=4362 --metrics-logging=true --lightpush=true --relay=true
2026-02-26 04:36:24.932 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.40.15 waku fc612be0e0196d892d411d0d1adf4fcd69f512ebeb52b0d6a1cd530334e2929a
2026-02-26 04:36:24.967 DEBUG [src.node.docker_mananger] Container started with ID fc612be0e019. Setting up logs at ./log/docker/receiving_node1_2026-02-26_04-36-24__f75efd15-da5d-4f2c-8b63-c01846825c67__wakuorg_nwaku:latest.log
2026-02-26 04:36:24.968 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 4358
2026-02-26 04:36:24.968 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-26 04:36:24.978 ERROR [src.node.docker_mananger] Max retries reached for container 51a31639fb84. Exiting log stream.
2026-02-26 04:36:25.535 ERROR [src.node.docker_mananger] Max retries reached for container 2fd2d0e3d700. Exiting log stream.
2026-02-26 04:36:25.968 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4358/health" -H "Content-Type: application/json" -d 'None'
2026-02-26 04:36:25.971 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-26 04:36:25.971 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-26 04:36:25.971 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4358/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-26 04:36:25.974 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.40.15/tcp/4359/p2p/16Uiu2HAm8PwgbR8UiFZXYATafvEGSQ8G7uUes13DCJTJKPBU7vie","/ip4/172.18.40.15/tcp/4360/ws/p2p/16Uiu2HAm8PwgbR8UiFZXYATafvEGSQ8G7uUes13DCJTJKPBU7vie"],"enrUri":"enr:-L24QOkD-PCRs9qtc4lPGoETRbqpPHBt2asmVcMMbQNWMVMzD_bmXKfi8zJtkcx2lJBha-sH7tL0R9QsZKqs6iKIDEMCgmlkgnY0gmlwhKwSKA-KbXVsdGlhZGRyc5YACASsEigPBhEHAAoErBIoDwYRCN0DgnJzhQADAQAAiXNlY3AyNTZrMaECwLaGPfrf4cSukOEEgW0bHJnubkeFucY4s0XeRigL70ODdGNwghEHg3VkcIIRCYV3YWt1Mgk"}'
2026-02-26 04:36:25.974 INFO [src.node.waku_node] REST service is ready !!
2026-02-26 04:36:25.981 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-26 04:36:25.982 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node2_2026-02-26_04-36-24__f75efd15-da5d-4f2c-8b63-c01846825c67__wakuorg_nwaku:latest.log
2026-02-26 04:36:25.982 DEBUG [src.node.waku_node] Starting Node...
2026-02-26 04:36:25.982 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-26 04:36:25.983 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-26 04:36:25.983 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.148.118
2026-02-26 04:36:25.984 DEBUG [src.node.docker_mananger] Generated ports ['1727', '1728', '1729', '1730', '1731']
2026-02-26 04:36:25.984 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-26 04:36:25.984 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-26 04:36:25.984 DEBUG [src.node.waku_node] Using volumes []
2026-02-26 04:36:25.984 DEBUG [src.node.docker_mananger] docker run -i -t -p 1727:1727 -p 1728:1728 -p 1729:1729 -p 1730:1730 -p 1731:1731 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=1729 --rest-port=1727 --tcp-port=1728 --discv5-udp-port=1730 --rest-address=0.0.0.0 --nat=extip:172.18.148.118 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=ae76b9ec769711af4c915a8eaefdb3b41a2d05a4ebdcfb27b91bd5ddda29ef06 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=1731 --metrics-logging=true --lightpush=false --relay=true
2026-02-26 04:36:26.179 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.148.118 waku 0937bc8a71a070431ef9624cba435271055216a7b6585b7128b80cc8a320a62d
2026-02-26 04:36:26.210 DEBUG [src.node.docker_mananger] Container started with ID 0937bc8a71a0. Setting up logs at ./log/docker/receiving_node2_2026-02-26_04-36-24__f75efd15-da5d-4f2c-8b63-c01846825c67__wakuorg_nwaku:latest.log
2026-02-26 04:36:26.211 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 1727
2026-02-26 04:36:26.211 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-26 04:36:27.212 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1727/health" -H "Content-Type: application/json" -d 'None'
2026-02-26 04:36:27.215 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-26 04:36:27.215 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-26 04:36:27.215 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1727/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-26 04:36:27.218 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.148.118/tcp/1728/p2p/16Uiu2HAmBYY5e6L8mUXkTnNHFxMS6ChPcMM7NTDq9jbMTg47Z9TT","/ip4/172.18.148.118/tcp/1729/ws/p2p/16Uiu2HAmBYY5e6L8mUXkTnNHFxMS6ChPcMM7NTDq9jbMTg47Z9TT"],"enrUri":"enr:-L24QH1_1rIw9Fv7S_i1Fa9OUDOA39qWI5vAA0_3ZiPHYbN1D0Ae6ne8qC6uE5KbOsCnXSbCK_Y-O9ikw6nUbAVlQf8CgmlkgnY0gmlwhKwSlHaKbXVsdGlhZGRyc5YACASsEpR2BgbAAAoErBKUdgYGwd0DgnJzhQADAQAAiXNlY3AyNTZrMaEC730f9Xh93D7gzVnBBvr5owgqIm_qbuzvNxwLQFNo-B6DdGNwggbAg3VkcIIGwoV3YWt1MgE"}'
2026-02-26 04:36:27.218 INFO [src.node.waku_node] REST service is ready !!
2026-02-26 04:36:27.218 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:1727/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.40.15/tcp/4359/p2p/16Uiu2HAm8PwgbR8UiFZXYATafvEGSQ8G7uUes13DCJTJKPBU7vie"]'
2026-02-26 04:36:27.255 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-26 04:36:27.264 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-26 04:36:27.265 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node1_2026-02-26_04-36-24__f75efd15-da5d-4f2c-8b63-c01846825c67__wakuorg_nwaku:latest.log
2026-02-26 04:36:27.265 DEBUG [src.node.waku_node] Starting Node...
2026-02-26 04:36:27.265 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-26 04:36:27.266 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-26 04:36:27.267 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.154.46
2026-02-26 04:36:27.267 DEBUG [src.node.docker_mananger] Generated ports ['41697', '41698', '41699', '41700', '41701']
2026-02-26 04:36:27.267 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-26 04:36:27.267 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-26 04:36:27.267 DEBUG [src.node.waku_node] Using volumes []
2026-02-26 04:36:27.267 DEBUG [src.node.docker_mananger] docker run -i -t -p 41697:41697 -p 41698:41698 -p 41699:41699 -p 41700:41700 -p 41701:41701 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=41699 --rest-port=41697 --tcp-port=41698 --discv5-udp-port=41700 --rest-address=0.0.0.0 --nat=extip:172.18.154.46 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=35d00cce43fb03b1bdee0f6c5f3bbffba9e3ddccefa7d2bba3c8d05704ad1b98 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=41701 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QOkD-PCRs9qtc4lPGoETRbqpPHBt2asmVcMMbQNWMVMzD_bmXKfi8zJtkcx2lJBha-sH7tL0R9QsZKqs6iKIDEMCgmlkgnY0gmlwhKwSKA-KbXVsdGlhZGRyc5YACASsEigPBhEHAAoErBIoDwYRCN0DgnJzhQADAQAAiXNlY3AyNTZrMaECwLaGPfrf4cSukOEEgW0bHJnubkeFucY4s0XeRigL70ODdGNwghEHg3VkcIIRCYV3YWt1Mgk --lightpushnode=/ip4/172.18.40.15/tcp/4359/p2p/16Uiu2HAm8PwgbR8UiFZXYATafvEGSQ8G7uUes13DCJTJKPBU7vie --lightpush=true --relay=true
2026-02-26 04:36:27.479 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.154.46 waku 5620add4ca029461f77145102ac680e9b7c1cd4e7408ab82bcad7b214d76da92
2026-02-26 04:36:27.511 DEBUG [src.node.docker_mananger] Container started with ID 5620add4ca02. Setting up logs at ./log/docker/lightpush_node1_2026-02-26_04-36-24__f75efd15-da5d-4f2c-8b63-c01846825c67__wakuorg_nwaku:latest.log
2026-02-26 04:36:27.511 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 41697
2026-02-26 04:36:27.511 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-26 04:36:28.512 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:41697/health" -H "Content-Type: application/json" -d 'None'
2026-02-26 04:36:28.524 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-26 04:36:28.525 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-26 04:36:28.525 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:41697/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-26 04:36:28.535 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.154.46/tcp/41698/p2p/16Uiu2HAmTt9CLdtXfy1vxgrhkVW6x6BrSYAyZGmWJnAhfWiRcWNR","/ip4/172.18.154.46/tcp/41699/ws/p2p/16Uiu2HAmTt9CLdtXfy1vxgrhkVW6x6BrSYAyZGmWJnAhfWiRcWNR"],"enrUri":"enr:-L24QILwj9-Gf0j9ZZMTOitiF4WdjCfAhud2qLb3aAtc_7pkH6d7P2weVmk1LzcjijUDSCdFvYZHiSu-Ff1QaPxwopYCgmlkgnY0gmlwhKwSmi6KbXVsdGlhZGRyc5YACASsEpouBqLiAAoErBKaLgai490DgnJzhQADAQAAiXNlY3AyNTZrMaED4j3bM6DyRQknYsImMU6LesMzVQV_q80vUJfoB1m4J8aDdGNwgqLig3VkcIKi5IV3YWt1Mgk"}'
2026-02-26 04:36:28.540 INFO [src.node.waku_node] REST service is ready !!
2026-02-26 04:36:28.540 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:41697/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.40.15/tcp/4359/p2p/16Uiu2HAm8PwgbR8UiFZXYATafvEGSQ8G7uUes13DCJTJKPBU7vie"]'
2026-02-26 04:36:28.564 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-26 04:36:28.565 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:41697/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.148.118/tcp/1728/p2p/16Uiu2HAmBYY5e6L8mUXkTnNHFxMS6ChPcMM7NTDq9jbMTg47Z9TT"]'
2026-02-26 04:36:28.589 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-26 04:36:28.591 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:4358/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-26 04:36:28.596 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-26 04:36:28.597 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:1727/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-26 04:36:28.605 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-26 04:36:28.606 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:41697/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-26 04:36:28.609 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-26 04:36:28.611 DEBUG [src.node.waku_node] Stopping container with id fc612be0e019
2026-02-26 04:36:29.150 DEBUG [src.node.waku_node] Container stopped.
2026-02-26 04:36:29.151 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:41697/lightpush/v1/message" -H "Content-Type: application/json" -d '{"pubsubTopic": "/waku/2/rs/3/0", "message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1772080588611492054}}'
2026-02-26 04:36:29.962 ERROR [src.node.docker_mananger] Max retries reached for container fc612be0e019. Exiting log stream.
2026-02-26 04:36:34.160 ERROR [src.node.api_clients.base_client] HTTP error occurred: 503 Server Error: Service Unavailable for url: http://127.0.0.1:41697/lightpush/v1/message. Response content: b'Failed to request a message push: dial_failure'
2026-02-26 04:36:34.162 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-02-26 04:36:34.163 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-02-26 04:36:34.163 DEBUG [src.node.waku_node] Stopping container with id 0937bc8a71a0
2026-02-26 04:36:34.705 DEBUG [src.node.waku_node] Container stopped.
2026-02-26 04:36:34.706 DEBUG [src.node.waku_node] Stopping container with id 5620add4ca02
2026-02-26 04:36:35.260 DEBUG [src.node.waku_node] Container stopped.
2026-02-26 04:36:35.263 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-02-26 04:36:35.271 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-26 04:36:35.279 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-26 04:36:35.286 DEBUG [src.node.docker_mananger] No errors found in the waku logs.