211 lines
36 KiB
Plaintext

2026-03-08 04:34:36.125 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-08 04:34:36.127 DEBUG [tests.conftest] Running test: test_unsubscribe_from_all_pubsub_topics_one_by_one with id: 2026-03-08_04-34-36__bf773cdc-1e2f-4cb3-90f9-fe165c06d6a3
2026-03-08 04:34:36.127 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-08 04:34:36.128 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-08 04:34:36.128 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-08 04:34:36.140 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-08 04:34:36.141 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-08_04-34-36__bf773cdc-1e2f-4cb3-90f9-fe165c06d6a3__wakuorg_nwaku:latest.log
2026-03-08 04:34:36.141 DEBUG [src.node.waku_node] Starting Node...
2026-03-08 04:34:36.141 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-08 04:34:36.144 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-08 04:34:36.144 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.86.27
2026-03-08 04:34:36.145 DEBUG [src.node.docker_mananger] Generated ports ['18744', '18745', '18746', '18747', '18748']
2026-03-08 04:34:36.145 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-08 04:34:36.145 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-08 04:34:36.147 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-08 04:34:36.147 DEBUG [src.node.waku_node] Using volumes []
2026-03-08 04:34:36.147 DEBUG [src.node.docker_mananger] docker run -i -t -p 18744:18744 -p 18745:18745 -p 18746:18746 -p 18747:18747 -p 18748:18748 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=18746 --rest-port=18744 --tcp-port=18745 --discv5-udp-port=18747 --rest-address=0.0.0.0 --nat=extip:172.18.86.27 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=f4a85eaaee4cbedeeeaeeea08d38d8492f826dfec4add1436cc3db172c84e3e6 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=18748 --metrics-logging=true --relay=true --filter=true
2026-03-08 04:34:36.361 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.86.27 waku dccfe565604bce6831727e8275f48f5012a28d1322af5fc659030a45d52d38e5
2026-03-08 04:34:36.391 ERROR [src.node.docker_mananger] Max retries reached for container af81f669e5ee. Exiting log stream.
2026-03-08 04:34:36.401 DEBUG [src.node.docker_mananger] Container started with ID dccfe565604b. Setting up logs at ./log/docker/node1_2026-03-08_04-34-36__bf773cdc-1e2f-4cb3-90f9-fe165c06d6a3__wakuorg_nwaku:latest.log
2026-03-08 04:34:36.402 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 18744
2026-03-08 04:34:36.402 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-08 04:34:37.046 ERROR [src.node.docker_mananger] Max retries reached for container c5fe27e36dd4. Exiting log stream.
2026-03-08 04:34:37.403 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:18744/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:37.406 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-08 04:34:37.507 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:18744/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:37.510 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-08 04:34:37.610 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:18744/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:37.618 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-08 04:34:37.618 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-08 04:34:37.619 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:18744/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:37.622 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.86.27/tcp/18745/p2p/16Uiu2HAm1xPvnb4uJ9bhTKsCTeRSktcTpc5QNc2szqhdbd6jos4j","/ip4/172.18.86.27/tcp/18746/ws/p2p/16Uiu2HAm1xPvnb4uJ9bhTKsCTeRSktcTpc5QNc2szqhdbd6jos4j"],"enrUri":"enr:-L24QNb3J3etbSyBKldfYDxMWwmTROt4DiPgMzLT06oAbdmbaiq3qrXiR56C4XCzDbInnJ4fFdXqTRnlt0plj-sCtZgCgmlkgnY0gmlwhKwSVhuKbXVsdGlhZGRyc5YACASsElYbBkk5AAoErBJWGwZJOt0DgnJzhQACAQAAiXNlY3AyNTZrMaECYQT6DVpvSSgM0cBzmalOF_x21XmK8U-aO89AiXdZV9CDdGNwgkk5g3VkcIJJO4V3YWt1MgU"}'
2026-03-08 04:34:37.622 INFO [src.node.waku_node] REST service is ready !!
2026-03-08 04:34:37.630 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-08 04:34:37.630 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-08_04-34-36__bf773cdc-1e2f-4cb3-90f9-fe165c06d6a3__wakuorg_nwaku:latest.log
2026-03-08 04:34:37.630 DEBUG [src.node.waku_node] Starting Node...
2026-03-08 04:34:37.630 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-08 04:34:37.632 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-08 04:34:37.632 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.196.36
2026-03-08 04:34:37.632 DEBUG [src.node.docker_mananger] Generated ports ['2424', '2425', '2426', '2427', '2428']
2026-03-08 04:34:37.633 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-08 04:34:37.633 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-08 04:34:37.633 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-08 04:34:37.633 DEBUG [src.node.waku_node] Using volumes []
2026-03-08 04:34:37.633 DEBUG [src.node.docker_mananger] docker run -i -t -p 2424:2424 -p 2425:2425 -p 2426:2426 -p 2427:2427 -p 2428:2428 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=2426 --rest-port=2424 --tcp-port=2425 --discv5-udp-port=2427 --rest-address=0.0.0.0 --nat=extip:172.18.196.36 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=df8debd8bcdfcaebcb6e7e7b2dbc0403374e7bec6df7260ad78c9b8b4cedf176 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=2428 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNb3J3etbSyBKldfYDxMWwmTROt4DiPgMzLT06oAbdmbaiq3qrXiR56C4XCzDbInnJ4fFdXqTRnlt0plj-sCtZgCgmlkgnY0gmlwhKwSVhuKbXVsdGlhZGRyc5YACASsElYbBkk5AAoErBJWGwZJOt0DgnJzhQACAQAAiXNlY3AyNTZrMaECYQT6DVpvSSgM0cBzmalOF_x21XmK8U-aO89AiXdZV9CDdGNwgkk5g3VkcIJJO4V3YWt1MgU
2026-03-08 04:34:37.852 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.196.36 waku af528dd580d161408307eb34b77cb02e9680ffb83ce7b92f04cf11744a87c4ee
2026-03-08 04:34:37.891 DEBUG [src.node.docker_mananger] Container started with ID af528dd580d1. Setting up logs at ./log/docker/node2_2026-03-08_04-34-36__bf773cdc-1e2f-4cb3-90f9-fe165c06d6a3__wakuorg_nwaku:latest.log
2026-03-08 04:34:37.891 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 2424
2026-03-08 04:34:37.892 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-08 04:34:38.892 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2424/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:38.895 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-08 04:34:38.995 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2424/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:39.001 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-08 04:34:39.102 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2424/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:39.105 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-08 04:34:39.105 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-08 04:34:39.105 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2424/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:39.108 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.196.36/tcp/2425/p2p/16Uiu2HAmF4ZABXFNXr2QWFfn4ttUCGS8HBTfhhk2K8XMsf9rgE3z","/ip4/172.18.196.36/tcp/2426/ws/p2p/16Uiu2HAmF4ZABXFNXr2QWFfn4ttUCGS8HBTfhhk2K8XMsf9rgE3z"],"enrUri":"enr:-L24QFUHsd78adOA3yQRPI7w-jUEiO0w4yUgG1KXVVDOmEzBdaYd6JzlMYVmLgKdW1BtCdoM8vFV-RFdL7D720WaNXICgmlkgnY0gmlwhKwSxCSKbXVsdGlhZGRyc5YACASsEsQkBgl5AAoErBLEJAYJet0DgnJzhQACAQAAiXNlY3AyNTZrMaEDI8DUQmGEj8U8KVc9CdbFfWi1Ovi01PPSMiDcife4qymDdGNwggl5g3VkcIIJe4V3YWt1MgE"}'
2026-03-08 04:34:39.108 INFO [src.node.waku_node] REST service is ready !!
2026-03-08 04:34:39.109 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:2424/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.86.27/tcp/18745/p2p/16Uiu2HAm1xPvnb4uJ9bhTKsCTeRSktcTpc5QNc2szqhdbd6jos4j"]'
2026-03-08 04:34:39.137 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:39.139 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:18744/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-08 04:34:39.145 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:39.147 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:2424/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-08 04:34:39.155 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:39.159 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:18744/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:34:39.171 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:39.171 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-08 04:34:39.272 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-08 04:34:39.272 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:18744/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:39.275 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772944479159644628,"ephemeral":false,"proof":""}]'
2026-03-08 04:34:39.278 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-08 04:34:39.278 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2424/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:39.284 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772944479159644628,"ephemeral":false,"proof":""}]'
2026-03-08 04:34:39.286 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:18744/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:34:39.295 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:39.296 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-08 04:34:39.397 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-08 04:34:39.397 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:18744/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:39.401 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772944479285986563,"ephemeral":false,"proof":""}]'
2026-03-08 04:34:39.403 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-08 04:34:39.403 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2424/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:39.405 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772944479285986563,"ephemeral":false,"proof":""}]'
2026-03-08 04:34:39.408 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:18744/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:34:39.417 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:39.418 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-08 04:34:39.518 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-08 04:34:39.519 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:18744/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:39.522 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772944479408292435,"ephemeral":false,"proof":""}]'
2026-03-08 04:34:39.524 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-08 04:34:39.524 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2424/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:39.527 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/newsService/1.0/weekly/protobuf","version":0,"timestamp":1772944478234714388,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772944479408292435,"ephemeral":false,"proof":""}]'
2026-03-08 04:34:39.571 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-08 04:34:39.572 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-08 04:34:39.572 DEBUG [src.node.waku_node] Stopping container with id dccfe565604b
2026-03-08 04:34:40.168 DEBUG [src.node.waku_node] Container stopped.
2026-03-08 04:34:40.169 DEBUG [src.node.waku_node] Stopping container with id af528dd580d1
2026-03-08 04:34:40.777 DEBUG [src.node.waku_node] Container stopped.
2026-03-08 04:34:40.780 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-08 04:34:40.813 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-08 04:34:40.849 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-08 04:34:40.850 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-08 04:34:40.850 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-08 04:34:40.851 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-08_04-34-36__bf773cdc-1e2f-4cb3-90f9-fe165c06d6a3__wakuorg_nwaku:latest.log
2026-03-08 04:34:40.852 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-08_04-34-36__bf773cdc-1e2f-4cb3-90f9-fe165c06d6a3__wakuorg_nwaku:latest.log
2026-03-08 04:34:40.857 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-08 04:34:40.857 DEBUG [tests.conftest] Running test: test_unsubscribe_from_all_pubsub_topics_one_by_one with id: 2026-03-08_04-34-40__7e62f172-a7bc-4920-8a2c-3999e9415a88
2026-03-08 04:34:40.857 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-08 04:34:40.857 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-08 04:34:40.858 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-08 04:34:40.866 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-08 04:34:40.866 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-08_04-34-40__7e62f172-a7bc-4920-8a2c-3999e9415a88__wakuorg_nwaku:latest.log
2026-03-08 04:34:40.867 DEBUG [src.node.waku_node] Starting Node...
2026-03-08 04:34:40.867 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-08 04:34:40.868 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-08 04:34:40.869 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.149.245
2026-03-08 04:34:40.869 DEBUG [src.node.docker_mananger] Generated ports ['38043', '38044', '38045', '38046', '38047']
2026-03-08 04:34:40.869 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-08 04:34:40.869 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-08 04:34:40.869 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-08 04:34:40.869 DEBUG [src.node.waku_node] Using volumes []
2026-03-08 04:34:40.869 DEBUG [src.node.docker_mananger] docker run -i -t -p 38043:38043 -p 38044:38044 -p 38045:38045 -p 38046:38046 -p 38047:38047 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=38045 --rest-port=38043 --tcp-port=38044 --discv5-udp-port=38046 --rest-address=0.0.0.0 --nat=extip:172.18.149.245 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=bd91927b5d2dbe1b02bebc7c756b4a9ee8ccd5d7349f3f5656507bf936c3ed47 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=38047 --metrics-logging=true --relay=true --filter=true
2026-03-08 04:34:41.053 ERROR [src.node.docker_mananger] Max retries reached for container dccfe565604b. Exiting log stream.
2026-03-08 04:34:41.070 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.149.245 waku db52868965f2676a9342c2bf87fe76c4dd67a9062841ab7e6b32182f6566a2cf
2026-03-08 04:34:41.107 DEBUG [src.node.docker_mananger] Container started with ID db52868965f2. Setting up logs at ./log/docker/node1_2026-03-08_04-34-40__7e62f172-a7bc-4920-8a2c-3999e9415a88__wakuorg_nwaku:latest.log
2026-03-08 04:34:41.107 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 38043
2026-03-08 04:34:41.107 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-08 04:34:41.626 ERROR [src.node.docker_mananger] Max retries reached for container af528dd580d1. Exiting log stream.
2026-03-08 04:34:42.108 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38043/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:42.111 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-08 04:34:42.211 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38043/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:42.214 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-08 04:34:42.315 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38043/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:42.318 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-08 04:34:42.318 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-08 04:34:42.318 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38043/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:42.320 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.149.245/tcp/38044/p2p/16Uiu2HAmH4SyunXZ83XfJS735MeZDkk5WoPoxdMjunUVG7bH7LCi","/ip4/172.18.149.245/tcp/38045/ws/p2p/16Uiu2HAmH4SyunXZ83XfJS735MeZDkk5WoPoxdMjunUVG7bH7LCi"],"enrUri":"enr:-L24QHyx_P6Uaex8XlSsDcWJRIuMkuuICFgks0ZGVaCfsxhSJZ833Mxv3VukdSiDovMdGewQMn6oL9kjtMhHMsvv7PcCgmlkgnY0gmlwhKwSlfWKbXVsdGlhZGRyc5YACASsEpX1BpScAAoErBKV9QaUnd0DgnJzhQACAQAAiXNlY3AyNTZrMaEDQXE38aECdTTEd2gy0vamJXvHzJvp2jOurTY0hqUZuoODdGNwgpScg3VkcIKUnoV3YWt1MgU"}'
2026-03-08 04:34:42.321 INFO [src.node.waku_node] REST service is ready !!
2026-03-08 04:34:42.328 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-08 04:34:42.328 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-08_04-34-40__7e62f172-a7bc-4920-8a2c-3999e9415a88__wakuorg_nwaku:latest.log
2026-03-08 04:34:42.328 DEBUG [src.node.waku_node] Starting Node...
2026-03-08 04:34:42.329 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-08 04:34:42.330 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-08 04:34:42.330 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.157.239
2026-03-08 04:34:42.330 DEBUG [src.node.docker_mananger] Generated ports ['13946', '13947', '13948', '13949', '13950']
2026-03-08 04:34:42.330 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-08 04:34:42.331 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-08 04:34:42.331 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-08 04:34:42.331 DEBUG [src.node.waku_node] Using volumes []
2026-03-08 04:34:42.331 DEBUG [src.node.docker_mananger] docker run -i -t -p 13946:13946 -p 13947:13947 -p 13948:13948 -p 13949:13949 -p 13950:13950 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=13948 --rest-port=13946 --tcp-port=13947 --discv5-udp-port=13949 --rest-address=0.0.0.0 --nat=extip:172.18.157.239 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=f1a7cbf8eea7bad96f9d5ce2543b55cc09eecbcdfdc767bcb2dbb651d3d9bff7 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=13950 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QHyx_P6Uaex8XlSsDcWJRIuMkuuICFgks0ZGVaCfsxhSJZ833Mxv3VukdSiDovMdGewQMn6oL9kjtMhHMsvv7PcCgmlkgnY0gmlwhKwSlfWKbXVsdGlhZGRyc5YACASsEpX1BpScAAoErBKV9QaUnd0DgnJzhQACAQAAiXNlY3AyNTZrMaEDQXE38aECdTTEd2gy0vamJXvHzJvp2jOurTY0hqUZuoODdGNwgpScg3VkcIKUnoV3YWt1MgU
2026-03-08 04:34:42.537 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.157.239 waku 6062bbe6e185a10986f76eb639a7f9169839b87c079e0fbe6a425a13da7ba0f3
2026-03-08 04:34:42.573 DEBUG [src.node.docker_mananger] Container started with ID 6062bbe6e185. Setting up logs at ./log/docker/node2_2026-03-08_04-34-40__7e62f172-a7bc-4920-8a2c-3999e9415a88__wakuorg_nwaku:latest.log
2026-03-08 04:34:42.573 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 13946
2026-03-08 04:34:42.573 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-08 04:34:43.574 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13946/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:43.577 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-08 04:34:43.677 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13946/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:43.687 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-08 04:34:43.788 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13946/health" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:43.791 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-08 04:34:43.791 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-08 04:34:43.791 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13946/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:43.793 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.157.239/tcp/13947/p2p/16Uiu2HAmCEUNV2EecGEwzLoqJ697SuvK71p9cLf8trr142ae9Zmh","/ip4/172.18.157.239/tcp/13948/ws/p2p/16Uiu2HAmCEUNV2EecGEwzLoqJ697SuvK71p9cLf8trr142ae9Zmh"],"enrUri":"enr:-L24QK4IPrV8YCqsYmbtN_LvN1dM3WcxRbWnLJzYfQq4GNtuJ8Gcz_Gx9AllMZ3558NfnJ4lIjQR1_O6RoUrYuaE5Q4CgmlkgnY0gmlwhKwSne-KbXVsdGlhZGRyc5YACASsEp3vBjZ7AAoErBKd7wY2fN0DgnJzhQACAQAAiXNlY3AyNTZrMaEC-bgpqiGUI_jMufBwLNAwGnJPGIY7zvzUCzF9WwiKuJCDdGNwgjZ7g3VkcII2fYV3YWt1MgE"}'
2026-03-08 04:34:43.793 INFO [src.node.waku_node] REST service is ready !!
2026-03-08 04:34:43.794 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13946/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.149.245/tcp/38044/p2p/16Uiu2HAmH4SyunXZ83XfJS735MeZDkk5WoPoxdMjunUVG7bH7LCi"]'
2026-03-08 04:34:43.816 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:43.820 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38043/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-08 04:34:43.828 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:43.829 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13946/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-08 04:34:43.835 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:43.839 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38043/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:34:43.850 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:43.851 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-08 04:34:43.952 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-08 04:34:43.953 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38043/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:43.958 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772944483838987975,"ephemeral":false,"proof":""}]'
2026-03-08 04:34:43.959 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-08 04:34:43.960 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13946/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:43.962 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772944483838987975,"ephemeral":false,"proof":""}]'
2026-03-08 04:34:43.964 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38043/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:34:43.973 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:43.974 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-08 04:34:44.075 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-08 04:34:44.076 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38043/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:44.079 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772944483964132156,"ephemeral":false,"proof":""}]'
2026-03-08 04:34:44.081 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-08 04:34:44.081 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13946/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:44.086 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772944483964132156,"ephemeral":false,"proof":""}]'
2026-03-08 04:34:44.089 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38043/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-08 04:34:44.103 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-08 04:34:44.104 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-08 04:34:44.204 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-08 04:34:44.205 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38043/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:44.209 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772944484089188239,"ephemeral":false,"proof":""}]'
2026-03-08 04:34:44.210 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-08 04:34:44.211 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13946/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d 'None'
2026-03-08 04:34:44.213 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/newsService/1.0/weekly/protobuf","version":0,"timestamp":1772944482559076712,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772944484089188239,"ephemeral":false,"proof":""}]'
2026-03-08 04:34:44.224 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-08 04:34:44.225 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-08 04:34:44.225 DEBUG [src.node.waku_node] Stopping container with id db52868965f2
2026-03-08 04:34:44.825 DEBUG [src.node.waku_node] Container stopped.
2026-03-08 04:34:44.828 DEBUG [src.node.waku_node] Stopping container with id 6062bbe6e185
2026-03-08 04:34:45.415 DEBUG [src.node.waku_node] Container stopped.
2026-03-08 04:34:45.417 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-08 04:34:45.459 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-08 04:34:45.495 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-08 04:34:45.497 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-08 04:34:45.497 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-08 04:34:45.497 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-08_04-34-40__7e62f172-a7bc-4920-8a2c-3999e9415a88__wakuorg_nwaku:latest.log
2026-03-08 04:34:45.498 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-08_04-34-40__7e62f172-a7bc-4920-8a2c-3999e9415a88__wakuorg_nwaku:latest.log