193 lines
34 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

2026-04-23 00:00:38.913 INFO [tests.conftest] Fleet bootstrap inactive pass --fleet (or set FLEET_BOOTSTRAP=true) to connect local nodes to the waku.test fleet
2026-04-23 00:00:38.913 DEBUG [tests.conftest] Running fixture setup: test_id
2026-04-23 00:00:38.914 DEBUG [tests.conftest] Running test: test_last_node_to_start_publishes with id: 2026-04-23_00-00-38__7a5422ae-978b-403d-9ebc-ef86fdda815c
2026-04-23 00:00:38.914 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-04-23 00:00:38.914 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-04-23 00:00:38.914 DEBUG [src.steps.relay] Running fixture setup: setup_main_relay_nodes
2026-04-23 00:00:38.920 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 00:00:38.921 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-04-23_00-00-38__7a5422ae-978b-403d-9ebc-ef86fdda815c__wakuorg_nwaku:latest.log
2026-04-23 00:00:38.921 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 00:00:38.921 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 00:00:38.922 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 00:00:38.922 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.93.236
2026-04-23 00:00:38.922 DEBUG [src.node.docker_mananger] Generated ports ['16897', '16898', '16899', '16900', '16901']
2026-04-23 00:00:38.922 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 00:00:38.922 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 00:00:38.922 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 00:00:38.923 DEBUG [src.node.docker_mananger] docker run -i -t -p 16897:16897 -p 16898:16898 -p 16899:16899 -p 16900:16900 -p 16901:16901 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=16899 --rest-port=16897 --tcp-port=16898 --discv5-udp-port=16900 --rest-address=0.0.0.0 --nat=extip:172.18.93.236 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=8bbbcc3c7befbb43604ceb88f9fea4eeb77ee093e90b0c0da72afd2ddbdae53c --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=16901 --metrics-logging=true --relay=true
2026-04-23 00:00:39.275 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.93.236 waku 96deb458274e06d569ac9f7cd876d03d6f6bf858ceb648ee457f60600f46fbd6
2026-04-23 00:00:39.301 ERROR [src.node.docker_mananger] Max retries reached for container 923664bae128. Exiting log stream.
2026-04-23 00:00:39.303 DEBUG [src.node.docker_mananger] Container started with ID 96deb458274e. Setting up logs at ./log/docker/node1_2026-04-23_00-00-38__7a5422ae-978b-403d-9ebc-ef86fdda815c__wakuorg_nwaku:latest.log
2026-04-23 00:00:39.304 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 16897
2026-04-23 00:00:39.305 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 00:00:39.708 ERROR [src.node.docker_mananger] Max retries reached for container 77551893078c. Exiting log stream.
2026-04-23 00:00:40.306 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16897/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:40.308 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 00:00:40.309 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 00:00:40.309 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16897/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:40.311 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.93.236/tcp/16898/p2p/16Uiu2HAkvM9wtbosQxtyDqt4oJ4ARHX1YuzwNK1j2JmbjLe7GkRW","/ip4/172.18.93.236/tcp/16899/ws/p2p/16Uiu2HAkvM9wtbosQxtyDqt4oJ4ARHX1YuzwNK1j2JmbjLe7GkRW"],"enrUri":"enr:-L24QPzMh5MKF8J7LvbqOxptT4DQWNbUaWdwiGIVt-l4uaOyUM_-xj2xm7ZjkulgmmVnaYPB9aSJj3EsFzZLwQwzVRQCgmlkgnY0gmlwhKwSXeyKbXVsdGlhZGRyc5YACASsEl3sBkICAAoErBJd7AZCA90DgnJzhQDGAQAAiXNlY3AyNTZrMaECDbNnzbAOV8mSvVUKAdjEIHuy4ez3vypflIUJpDyHhzGDdGNwgkICg3VkcIJCBIV3YWt1MgE"}'
2026-04-23 00:00:40.311 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 00:00:40.317 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 00:00:40.317 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-04-23_00-00-38__7a5422ae-978b-403d-9ebc-ef86fdda815c__wakuorg_nwaku:latest.log
2026-04-23 00:00:40.317 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 00:00:40.317 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 00:00:40.318 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 00:00:40.319 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.102.195
2026-04-23 00:00:40.319 DEBUG [src.node.docker_mananger] Generated ports ['6762', '6763', '6764', '6765', '6766']
2026-04-23 00:00:40.319 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 00:00:40.319 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 00:00:40.319 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 00:00:40.319 DEBUG [src.node.docker_mananger] docker run -i -t -p 6762:6762 -p 6763:6763 -p 6764:6764 -p 6765:6765 -p 6766:6766 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=6764 --rest-port=6762 --tcp-port=6763 --discv5-udp-port=6765 --rest-address=0.0.0.0 --nat=extip:172.18.102.195 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=d1dd0af69bb0eb9a3e18bc1ceda2303d02b024ce38fa4d1aadbf28ba4abcae2d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=6766 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPzMh5MKF8J7LvbqOxptT4DQWNbUaWdwiGIVt-l4uaOyUM_-xj2xm7ZjkulgmmVnaYPB9aSJj3EsFzZLwQwzVRQCgmlkgnY0gmlwhKwSXeyKbXVsdGlhZGRyc5YACASsEl3sBkICAAoErBJd7AZCA90DgnJzhQDGAQAAiXNlY3AyNTZrMaECDbNnzbAOV8mSvVUKAdjEIHuy4ez3vypflIUJpDyHhzGDdGNwgkICg3VkcIJCBIV3YWt1MgE
2026-04-23 00:00:40.564 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.102.195 waku 4e122ad553a7781fb86d031198795b504292097d99dd6b783fbface6abb8628d
2026-04-23 00:00:40.595 DEBUG [src.node.docker_mananger] Container started with ID 4e122ad553a7. Setting up logs at ./log/docker/node2_2026-04-23_00-00-38__7a5422ae-978b-403d-9ebc-ef86fdda815c__wakuorg_nwaku:latest.log
2026-04-23 00:00:40.595 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 6762
2026-04-23 00:00:40.595 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 00:00:41.595 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6762/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:41.600 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 00:00:41.601 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 00:00:41.601 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6762/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:41.603 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.102.195/tcp/6763/p2p/16Uiu2HAkuZfVgr7y5gFZKL8vm5t8J9pK6EmB7JtzQkaig3EwTFNM","/ip4/172.18.102.195/tcp/6764/ws/p2p/16Uiu2HAkuZfVgr7y5gFZKL8vm5t8J9pK6EmB7JtzQkaig3EwTFNM"],"enrUri":"enr:-L24QNk9orO_CD-AJyFjcEuCdNhp0buAmKjfv6qgpkiLzZkLBcxs3AE28AoQJCaRIKX-3RYiOWWf9H_P62UixKgFYjoCgmlkgnY0gmlwhKwSZsOKbXVsdGlhZGRyc5YACASsEmbDBhprAAoErBJmwwYabN0DgnJzhQDGAQAAiXNlY3AyNTZrMaECAgwZOuyw5tGy8JWaOMOJAozFJOR8gLpjb1TIfytT_F6DdGNwghprg3VkcIIabYV3YWt1MgE"}'
2026-04-23 00:00:41.603 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 00:00:41.603 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6762/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.93.236/tcp/16898/p2p/16Uiu2HAkvM9wtbosQxtyDqt4oJ4ARHX1YuzwNK1j2JmbjLe7GkRW"]'
2026-04-23 00:00:41.605 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 00:00:41.606 DEBUG [src.steps.relay] Running fixture setup: setup_optional_relay_nodes
2026-04-23 00:00:41.611 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 00:00:41.612 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-04-23_00-00-38__7a5422ae-978b-403d-9ebc-ef86fdda815c__wakuorg_nwaku:latest.log
2026-04-23 00:00:41.612 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 00:00:41.612 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 00:00:41.613 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 00:00:41.613 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.22.200
2026-04-23 00:00:41.613 DEBUG [src.node.docker_mananger] Generated ports ['32241', '32242', '32243', '32244', '32245']
2026-04-23 00:00:41.613 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 00:00:41.613 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 00:00:41.613 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 00:00:41.613 DEBUG [src.node.docker_mananger] docker run -i -t -p 32241:32241 -p 32242:32242 -p 32243:32243 -p 32244:32244 -p 32245:32245 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=32243 --rest-port=32241 --tcp-port=32242 --discv5-udp-port=32244 --rest-address=0.0.0.0 --nat=extip:172.18.22.200 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=fffb232d18ccff2bbca9d7cfc7e0e25c886bf8cf7be515a6e76c8eadd24b596b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=32245 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPzMh5MKF8J7LvbqOxptT4DQWNbUaWdwiGIVt-l4uaOyUM_-xj2xm7ZjkulgmmVnaYPB9aSJj3EsFzZLwQwzVRQCgmlkgnY0gmlwhKwSXeyKbXVsdGlhZGRyc5YACASsEl3sBkICAAoErBJd7AZCA90DgnJzhQDGAQAAiXNlY3AyNTZrMaECDbNnzbAOV8mSvVUKAdjEIHuy4ez3vypflIUJpDyHhzGDdGNwgkICg3VkcIJCBIV3YWt1MgE
2026-04-23 00:00:42.338 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.22.200 waku 2b19773987abb6eac0cd7c93cb591c5ac8fbaf718609c8cc20f4ef0a82cbd31f
2026-04-23 00:00:42.590 DEBUG [src.node.docker_mananger] Container started with ID 2b19773987ab. Setting up logs at ./log/docker/node3_2026-04-23_00-00-38__7a5422ae-978b-403d-9ebc-ef86fdda815c__wakuorg_nwaku:latest.log
2026-04-23 00:00:42.590 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 32241
2026-04-23 00:00:42.590 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 00:00:43.591 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32241/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:43.594 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 00:00:43.594 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 00:00:43.594 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32241/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:43.596 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.22.200/tcp/32242/p2p/16Uiu2HAm5Yubwx5cZEpaK9i5QBst5FmXCtywBn94hSdRMJCgKowU","/ip4/172.18.22.200/tcp/32243/ws/p2p/16Uiu2HAm5Yubwx5cZEpaK9i5QBst5FmXCtywBn94hSdRMJCgKowU"],"enrUri":"enr:-L24QKDsFJSxHfm0UjysiN9n88Bksi5QHeWcZsbxbn5fuTt7EqU0G6efBpNy4m58XxkQ3ryT-tZbxZm8fXXJZBeWIuQCgmlkgnY0gmlwhKwSFsiKbXVsdGlhZGRyc5YACASsEhbIBn3yAAoErBIWyAZ9890DgnJzhQDGAQAAiXNlY3AyNTZrMaEClm9WwoojP4rgN_cDOLUngmQNyurxGzeL296wv9V3ey-DdGNwgn3yg3VkcIJ99IV3YWt1MgE"}'
2026-04-23 00:00:43.596 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 00:00:43.596 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32241/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.93.236/tcp/16898/p2p/16Uiu2HAkvM9wtbosQxtyDqt4oJ4ARHX1YuzwNK1j2JmbjLe7GkRW"]'
2026-04-23 00:00:43.598 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 00:00:43.604 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 00:00:43.604 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-04-23_00-00-38__7a5422ae-978b-403d-9ebc-ef86fdda815c__wakuorg_nwaku:latest.log
2026-04-23 00:00:43.604 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 00:00:43.604 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 00:00:43.605 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 00:00:43.606 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.193.170
2026-04-23 00:00:43.606 DEBUG [src.node.docker_mananger] Generated ports ['13986', '13987', '13988', '13989', '13990']
2026-04-23 00:00:43.606 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 00:00:43.606 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 00:00:43.606 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 00:00:43.606 DEBUG [src.node.docker_mananger] docker run -i -t -p 13986:13986 -p 13987:13987 -p 13988:13988 -p 13989:13989 -p 13990:13990 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=13988 --rest-port=13986 --tcp-port=13987 --discv5-udp-port=13989 --rest-address=0.0.0.0 --nat=extip:172.18.193.170 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=c397d5d9cb69a4d8ecc155cecbc1bbffdcf33ba861cda6d0e1f051768f078cff --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=13990 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPzMh5MKF8J7LvbqOxptT4DQWNbUaWdwiGIVt-l4uaOyUM_-xj2xm7ZjkulgmmVnaYPB9aSJj3EsFzZLwQwzVRQCgmlkgnY0gmlwhKwSXeyKbXVsdGlhZGRyc5YACASsEl3sBkICAAoErBJd7AZCA90DgnJzhQDGAQAAiXNlY3AyNTZrMaECDbNnzbAOV8mSvVUKAdjEIHuy4ez3vypflIUJpDyHhzGDdGNwgkICg3VkcIJCBIV3YWt1MgE
2026-04-23 00:00:44.104 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.193.170 waku 5a12a20bd607be1a481358d733493fa5bcf2139bf07e6b172006af5a9afbff17
2026-04-23 00:00:44.130 DEBUG [src.node.docker_mananger] Container started with ID 5a12a20bd607. Setting up logs at ./log/docker/node4_2026-04-23_00-00-38__7a5422ae-978b-403d-9ebc-ef86fdda815c__wakuorg_nwaku:latest.log
2026-04-23 00:00:44.130 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 13986
2026-04-23 00:00:44.130 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 00:00:45.130 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13986/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:45.133 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 00:00:45.133 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 00:00:45.133 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13986/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:45.135 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.193.170/tcp/13987/p2p/16Uiu2HAmKfXTquQMNxSe9SSCRZQSgRujZtEQ58pVrP8wj9N1mLVW","/ip4/172.18.193.170/tcp/13988/ws/p2p/16Uiu2HAmKfXTquQMNxSe9SSCRZQSgRujZtEQ58pVrP8wj9N1mLVW"],"enrUri":"enr:-L24QHqasbkd6SK1TosOCdkRUHwSv70pYklmqCXuYfL2YjwjRGI_Snfv9UdmSzclEMihxry1UG-M4BVeLhBZ6T4tUZYCgmlkgnY0gmlwhKwSwaqKbXVsdGlhZGRyc5YACASsEsGqBjajAAoErBLBqgY2pN0DgnJzhQDGAQAAiXNlY3AyNTZrMaEDaCT88gIMw61srFBCVfqDc_UuiSlL_9anem13LGdZfCGDdGNwgjajg3VkcII2pYV3YWt1MgE"}'
2026-04-23 00:00:45.135 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 00:00:45.135 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13986/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.93.236/tcp/16898/p2p/16Uiu2HAkvM9wtbosQxtyDqt4oJ4ARHX1YuzwNK1j2JmbjLe7GkRW"]'
2026-04-23 00:00:45.137 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 00:00:45.144 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 00:00:45.144 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-04-23_00-00-38__7a5422ae-978b-403d-9ebc-ef86fdda815c__wakuorg_nwaku:latest.log
2026-04-23 00:00:45.144 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 00:00:45.144 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 00:00:45.145 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 00:00:45.145 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.89.8
2026-04-23 00:00:45.145 DEBUG [src.node.docker_mananger] Generated ports ['56320', '56321', '56322', '56323', '56324']
2026-04-23 00:00:45.145 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 00:00:45.146 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 00:00:45.146 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 00:00:45.146 DEBUG [src.node.docker_mananger] docker run -i -t -p 56320:56320 -p 56321:56321 -p 56322:56322 -p 56323:56323 -p 56324:56324 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=56322 --rest-port=56320 --tcp-port=56321 --discv5-udp-port=56323 --rest-address=0.0.0.0 --nat=extip:172.18.89.8 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=0f331f8db5cf7c4f5fbab5ffbbb1ebfa6bc5ae2a3c8352dd2d65b3254fa86feb --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=56324 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPzMh5MKF8J7LvbqOxptT4DQWNbUaWdwiGIVt-l4uaOyUM_-xj2xm7ZjkulgmmVnaYPB9aSJj3EsFzZLwQwzVRQCgmlkgnY0gmlwhKwSXeyKbXVsdGlhZGRyc5YACASsEl3sBkICAAoErBJd7AZCA90DgnJzhQDGAQAAiXNlY3AyNTZrMaECDbNnzbAOV8mSvVUKAdjEIHuy4ez3vypflIUJpDyHhzGDdGNwgkICg3VkcIJCBIV3YWt1MgE
2026-04-23 00:00:45.446 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.89.8 waku 5e1bb8be35d97561f10a31b372a7d9a463987a9b3c732edceae33f83152cc470
2026-04-23 00:00:45.482 DEBUG [src.node.docker_mananger] Container started with ID 5e1bb8be35d9. Setting up logs at ./log/docker/node5_2026-04-23_00-00-38__7a5422ae-978b-403d-9ebc-ef86fdda815c__wakuorg_nwaku:latest.log
2026-04-23 00:00:45.482 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 56320
2026-04-23 00:00:45.482 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 00:00:46.484 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56320/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:46.487 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 00:00:46.488 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 00:00:46.488 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56320/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:46.490 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.89.8/tcp/56321/p2p/16Uiu2HAm71UwJMSGausAEzQDurvh4T6gSb4Qh9iuhHEU2GWRZ88o","/ip4/172.18.89.8/tcp/56322/ws/p2p/16Uiu2HAm71UwJMSGausAEzQDurvh4T6gSb4Qh9iuhHEU2GWRZ88o"],"enrUri":"enr:-L24QKg3gN7Ec_AA8G_EKXy6Csy6vn1dUkJdK2ZMmzQkO7mGd3TfK_MkIgzlB5mHy7rZC25SjaPoc7Hts8N0bA2p1I4CgmlkgnY0gmlwhKwSWQiKbXVsdGlhZGRyc5YACASsElkIBtwBAAoErBJZCAbcAt0DgnJzhQDGAQAAiXNlY3AyNTZrMaECrBnRPbUe11VwTgfk9YHQ0Y0gi7x8GwbdE6dKES5UZKCDdGNwgtwBg3VkcILcA4V3YWt1MgE"}'
2026-04-23 00:00:46.490 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 00:00:46.490 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56320/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.93.236/tcp/16898/p2p/16Uiu2HAkvM9wtbosQxtyDqt4oJ4ARHX1YuzwNK1j2JmbjLe7GkRW"]'
2026-04-23 00:00:46.492 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 00:00:46.493 DEBUG [src.steps.relay] Running fixture setup: subscribe_main_relay_nodes
2026-04-23 00:00:46.493 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16897/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/1"]'
2026-04-23 00:00:46.500 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 00:00:46.500 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6762/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/1"]'
2026-04-23 00:00:46.505 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 00:00:46.506 DEBUG [src.steps.relay] Running fixture setup: subscribe_optional_relay_nodes
2026-04-23 00:00:46.506 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32241/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/1"]'
2026-04-23 00:00:46.512 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 00:00:46.513 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13986/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/1"]'
2026-04-23 00:00:46.517 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 00:00:46.518 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56320/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/1"]'
2026-04-23 00:00:46.522 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 00:00:46.523 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16897/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-04-23 00:00:46.530 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 00:00:46.531 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-04-23 00:00:46.631 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-04-23 00:00:46.631 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16897/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:46.634 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776902446522907648,"ephemeral":false,"proof":""}]'
2026-04-23 00:00:46.635 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-04-23 00:00:46.635 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6762/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:46.637 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776902446522907648,"ephemeral":false,"proof":""}]'
2026-04-23 00:00:46.638 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-04-23 00:00:46.638 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32241/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:46.640 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776902446522907648,"ephemeral":false,"proof":""}]'
2026-04-23 00:00:46.641 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-04-23 00:00:46.641 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13986/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:46.643 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776902446522907648,"ephemeral":false,"proof":""}]'
2026-04-23 00:00:46.644 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-04-23 00:00:46.644 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56320/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:46.646 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776902446522907648,"ephemeral":false,"proof":""}]'
2026-04-23 00:00:46.647 INFO [src.steps.relay] WARM UP successful!!
2026-04-23 00:00:46.648 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56320/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-04-23 00:00:46.652 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 00:00:46.652 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-04-23 00:00:46.752 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-04-23 00:00:46.753 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16897/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:46.755 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776902446648651322,"ephemeral":false,"proof":""}]'
2026-04-23 00:00:46.756 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-04-23 00:00:46.756 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6762/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:46.758 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776902446648651322,"ephemeral":false,"proof":""}]'
2026-04-23 00:00:46.759 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-04-23 00:00:46.759 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32241/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:46.761 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776902446648651322,"ephemeral":false,"proof":""}]'
2026-04-23 00:00:46.762 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-04-23 00:00:46.762 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13986/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:46.764 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776902446648651322,"ephemeral":false,"proof":""}]'
2026-04-23 00:00:46.765 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-04-23 00:00:46.765 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56320/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F1" -H "Content-Type: application/json" -d 'None'
2026-04-23 00:00:46.767 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1776902446648651322,"ephemeral":false,"proof":""}]'
2026-04-23 00:00:46.770 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-04-23 00:00:46.771 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-04-23 00:00:46.771 DEBUG [src.node.waku_node] Stopping container with id 96deb458274e
2026-04-23 00:00:47.304 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 00:00:47.304 DEBUG [src.node.waku_node] Stopping container with id 4e122ad553a7
2026-04-23 00:00:47.849 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 00:00:47.850 DEBUG [src.node.waku_node] Stopping container with id 2b19773987ab
2026-04-23 00:00:48.114 ERROR [src.node.docker_mananger] Max retries reached for container 96deb458274e. Exiting log stream.
2026-04-23 00:00:48.679 ERROR [src.node.docker_mananger] Max retries reached for container 4e122ad553a7. Exiting log stream.
2026-04-23 00:00:48.903 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 00:00:48.903 DEBUG [src.node.waku_node] Stopping container with id 5a12a20bd607
2026-04-23 00:00:49.362 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 00:00:49.363 DEBUG [src.node.waku_node] Stopping container with id 5e1bb8be35d9
2026-04-23 00:00:49.713 ERROR [src.node.docker_mananger] Max retries reached for container 2b19773987ab. Exiting log stream.
2026-04-23 00:00:49.991 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 00:00:49.992 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-04-23 00:00:50.010 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 00:00:50.021 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 00:00:50.027 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 00:00:50.033 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 00:00:50.038 DEBUG [src.node.docker_mananger] No errors found in the waku logs.