107 lines
18 KiB
Plaintext

2025-12-11 04:14:21.207 DEBUG [tests.conftest] Running fixture setup: test_id
2025-12-11 04:14:21.208 DEBUG [tests.conftest] Running test: test_publish_after_node_pauses_and_pauses with id: 2025-12-11_04-14-21__c305e526-b748-4248-83b9-f7134f453554
2025-12-11 04:14:21.208 DEBUG [src.steps.common] Running fixture setup: common_setup
2025-12-11 04:14:21.208 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2025-12-11 04:14:21.208 DEBUG [src.steps.relay] Running fixture setup: setup_main_relay_nodes
2025-12-11 04:14:21.214 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-11 04:14:21.214 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2025-12-11_04-14-21__c305e526-b748-4248-83b9-f7134f453554__wakuorg_nwaku:latest.log
2025-12-11 04:14:21.214 DEBUG [src.node.waku_node] Starting Node...
2025-12-11 04:14:21.214 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-11 04:14:21.215 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-11 04:14:21.216 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.228.71
2025-12-11 04:14:21.216 DEBUG [src.node.docker_mananger] Generated ports ['32614', '32615', '32616', '32617', '32618']
2025-12-11 04:14:21.216 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-11 04:14:21.216 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-11 04:14:21.216 DEBUG [src.node.waku_node] Using volumes []
2025-12-11 04:14:21.216 DEBUG [src.node.docker_mananger] docker run -i -t -p 32614:32614 -p 32615:32615 -p 32616:32616 -p 32617:32617 -p 32618:32618 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=32616 --rest-port=32614 --tcp-port=32615 --discv5-udp-port=32617 --rest-address=0.0.0.0 --nat=extip:172.18.228.71 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=c39ad0bbcbf46eae94cf85e1201e16b17ebf8dce8b280b0c64e1eaad8f6fbdb1 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=32618 --metrics-logging=true --relay=true
2025-12-11 04:14:21.369 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.228.71 waku 2d25d3336ccc97ffab1326fa87b506890e85fa71ad71854fdb4194f0ed22713a
2025-12-11 04:14:21.394 DEBUG [src.node.docker_mananger] Container started with ID 2d25d3336ccc. Setting up logs at ./log/docker/node1_2025-12-11_04-14-21__c305e526-b748-4248-83b9-f7134f453554__wakuorg_nwaku:latest.log
2025-12-11 04:14:21.395 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 32614
2025-12-11 04:14:21.395 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-11 04:14:21.536 ERROR [src.node.docker_mananger] Max retries reached for container b23b988d018b. Exiting log stream.
2025-12-11 04:14:22.006 ERROR [src.node.docker_mananger] Max retries reached for container ae4e18bedd44. Exiting log stream.
2025-12-11 04:14:22.396 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32614/health" -H "Content-Type: application/json" -d 'None'
2025-12-11 04:14:22.399 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-11 04:14:22.399 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-11 04:14:22.399 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32614/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-11 04:14:22.402 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.228.71/tcp/32615/p2p/16Uiu2HAm7TqAu7FQ2XC8r4GaPf5xAAKa5J5PWpTexxCLhhrr43ZC","/ip4/172.18.228.71/tcp/32616/ws/p2p/16Uiu2HAm7TqAu7FQ2XC8r4GaPf5xAAKa5J5PWpTexxCLhhrr43ZC"],"enrUri":"enr:-L24QMs_deWzYzjjrUPWOKmrF4mMLxOAbPQ2xgeJVe2cFWgRMCGx2U8NhWbNMHjomXhW3461M2nsY-6YeJ820cv6HQcCgmlkgnY0gmlwhKwS5EeKbXVsdGlhZGRyc5YACASsEuRHBn9nAAoErBLkRwZ_aN0DgnJzhQADAQAAiXNlY3AyNTZrMaECstnMLaNgLkRXHOhXOZGXoVd07NRB04rW4L7zYFs45ruDdGNwgn9ng3VkcIJ_aYV3YWt1MgE"}'
2025-12-11 04:14:22.402 INFO [src.node.waku_node] REST service is ready !!
2025-12-11 04:14:22.408 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-11 04:14:22.408 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2025-12-11_04-14-21__c305e526-b748-4248-83b9-f7134f453554__wakuorg_nwaku:latest.log
2025-12-11 04:14:22.408 DEBUG [src.node.waku_node] Starting Node...
2025-12-11 04:14:22.408 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-11 04:14:22.409 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-11 04:14:22.410 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.236.202
2025-12-11 04:14:22.410 DEBUG [src.node.docker_mananger] Generated ports ['4782', '4783', '4784', '4785', '4786']
2025-12-11 04:14:22.410 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-11 04:14:22.410 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-11 04:14:22.410 DEBUG [src.node.waku_node] Using volumes []
2025-12-11 04:14:22.410 DEBUG [src.node.docker_mananger] docker run -i -t -p 4782:4782 -p 4783:4783 -p 4784:4784 -p 4785:4785 -p 4786:4786 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=4784 --rest-port=4782 --tcp-port=4783 --discv5-udp-port=4785 --rest-address=0.0.0.0 --nat=extip:172.18.236.202 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=8b69c1eb12fb72f26afe9d8cfa758e6bab79aa6c2eaff81e12cf3abf80e8c9c8 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=4786 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QMs_deWzYzjjrUPWOKmrF4mMLxOAbPQ2xgeJVe2cFWgRMCGx2U8NhWbNMHjomXhW3461M2nsY-6YeJ820cv6HQcCgmlkgnY0gmlwhKwS5EeKbXVsdGlhZGRyc5YACASsEuRHBn9nAAoErBLkRwZ_aN0DgnJzhQADAQAAiXNlY3AyNTZrMaECstnMLaNgLkRXHOhXOZGXoVd07NRB04rW4L7zYFs45ruDdGNwgn9ng3VkcIJ_aYV3YWt1MgE
2025-12-11 04:14:22.592 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.236.202 waku 88ef891dd161076993f3f7c86d67f881f05bdf383a12ecc38a408f2c1742f9c1
2025-12-11 04:14:22.616 DEBUG [src.node.docker_mananger] Container started with ID 88ef891dd161. Setting up logs at ./log/docker/node2_2025-12-11_04-14-21__c305e526-b748-4248-83b9-f7134f453554__wakuorg_nwaku:latest.log
2025-12-11 04:14:22.616 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 4782
2025-12-11 04:14:22.616 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-11 04:14:23.617 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4782/health" -H "Content-Type: application/json" -d 'None'
2025-12-11 04:14:23.629 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-11 04:14:23.629 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-11 04:14:23.632 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4782/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-11 04:14:23.636 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.236.202/tcp/4783/p2p/16Uiu2HAkyR51GjUgddvhogBRVuJ4VXLeH8irKiiF3qL2pyhvyxTe","/ip4/172.18.236.202/tcp/4784/ws/p2p/16Uiu2HAkyR51GjUgddvhogBRVuJ4VXLeH8irKiiF3qL2pyhvyxTe"],"enrUri":"enr:-L24QB9STAG3tB4SAlcuy5QtaYQ9ashkwef5bTJg59gCoy7VP31kz85P3ihQOLY0FDPNKsdbnVd-vBidSlpj30ENTHoCgmlkgnY0gmlwhKwS7MqKbXVsdGlhZGRyc5YACASsEuzKBhKvAAoErBLsygYSsN0DgnJzhQADAQAAiXNlY3AyNTZrMaECO0czr_A2d9qp4EN9JOFh-f6LyCUw3Erd8oY2Q_Eb8VWDdGNwghKvg3VkcIISsYV3YWt1MgE"}'
2025-12-11 04:14:23.637 INFO [src.node.waku_node] REST service is ready !!
2025-12-11 04:14:23.637 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:4782/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.228.71/tcp/32615/p2p/16Uiu2HAm7TqAu7FQ2XC8r4GaPf5xAAKa5J5PWpTexxCLhhrr43ZC"]'
2025-12-11 04:14:23.640 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-11 04:14:23.640 DEBUG [src.steps.relay] Running fixture setup: subscribe_main_relay_nodes
2025-12-11 04:14:23.640 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32614/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2025-12-11 04:14:23.643 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-11 04:14:23.643 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:4782/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2025-12-11 04:14:23.647 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-11 04:14:23.649 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32614/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2025-12-11 04:14:23.652 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-11 04:14:23.654 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2025-12-11 04:14:23.754 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2025-12-11 04:14:23.754 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32614/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-11 04:14:23.757 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1765426463649340416,"ephemeral":false,"proof":""}]'
2025-12-11 04:14:23.759 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2025-12-11 04:14:23.759 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4782/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-11 04:14:23.761 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1765426463649340416,"ephemeral":false,"proof":""}]'
2025-12-11 04:14:23.763 INFO [src.steps.relay] WARM UP successful!!
2025-12-11 04:14:23.764 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32614/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2025-12-11 04:14:23.767 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-11 04:14:23.767 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2025-12-11 04:14:23.868 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2025-12-11 04:14:23.868 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32614/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-11 04:14:23.871 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1765426463764006385,"ephemeral":false,"proof":""}]'
2025-12-11 04:14:23.873 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2025-12-11 04:14:23.873 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4782/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-11 04:14:23.875 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1765426463764006385,"ephemeral":false,"proof":""}]'
2025-12-11 04:14:23.877 DEBUG [src.node.waku_node] Pausing container with id 2d25d3336ccc
2025-12-11 04:14:23.886 DEBUG [src.node.waku_node] Unpause container with id 2d25d3336ccc
2025-12-11 04:14:23.895 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32614/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "TTE=", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2025-12-11 04:14:23.899 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-11 04:14:23.899 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2025-12-11 04:14:23.999 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2025-12-11 04:14:24.000 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32614/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-11 04:14:24.002 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TTE=","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1765426463895069001,"ephemeral":false,"proof":""}]'
2025-12-11 04:14:24.004 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2025-12-11 04:14:24.004 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4782/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-11 04:14:24.006 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TTE=","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1765426463895069001,"ephemeral":false,"proof":""}]'
2025-12-11 04:14:24.008 DEBUG [src.node.waku_node] Pausing container with id 88ef891dd161
2025-12-11 04:14:24.017 DEBUG [src.node.waku_node] Unpause container with id 88ef891dd161
2025-12-11 04:14:24.025 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32614/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "TTI=", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2025-12-11 04:14:24.029 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-11 04:14:24.029 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2025-12-11 04:14:24.129 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2025-12-11 04:14:24.130 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32614/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-11 04:14:24.132 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TTI=","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1765426464025279700,"ephemeral":false,"proof":""}]'
2025-12-11 04:14:24.134 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2025-12-11 04:14:24.134 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4782/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-11 04:14:24.137 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TTI=","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1765426464025279700,"ephemeral":false,"proof":""}]'
2025-12-11 04:14:24.140 DEBUG [tests.conftest] Running fixture teardown: test_setup
2025-12-11 04:14:24.141 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2025-12-11 04:14:24.141 DEBUG [src.node.waku_node] Stopping container with id 2d25d3336ccc
2025-12-11 04:14:24.610 DEBUG [src.node.waku_node] Container stopped.
2025-12-11 04:14:24.610 DEBUG [src.node.waku_node] Stopping container with id 88ef891dd161
2025-12-11 04:14:25.063 DEBUG [src.node.waku_node] Container stopped.
2025-12-11 04:14:25.065 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2025-12-11 04:14:25.071 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-11 04:14:25.076 DEBUG [src.node.docker_mananger] No errors found in the waku logs.