163 lines
30 KiB
Plaintext

2026-03-05 04:33:25.390 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-05 04:33:25.391 DEBUG [tests.conftest] Running test: test_optional_nodes_not_subscribed_to_same_pubsub_topic with id: 2026-03-05_04-33-25__92661b1f-c157-4ecb-b0d5-b480081a8057
2026-03-05 04:33:25.391 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-05 04:33:25.391 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-05 04:33:25.391 DEBUG [src.steps.relay] Running fixture setup: setup_main_relay_nodes
2026-03-05 04:33:25.399 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:33:25.399 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-05_04-33-25__92661b1f-c157-4ecb-b0d5-b480081a8057__wakuorg_nwaku:latest.log
2026-03-05 04:33:25.399 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:33:25.399 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:33:25.400 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:33:25.401 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.25.7
2026-03-05 04:33:25.401 DEBUG [src.node.docker_mananger] Generated ports ['41562', '41563', '41564', '41565', '41566']
2026-03-05 04:33:25.401 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:33:25.401 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:33:25.401 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:33:25.401 DEBUG [src.node.docker_mananger] docker run -i -t -p 41562:41562 -p 41563:41563 -p 41564:41564 -p 41565:41565 -p 41566:41566 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=41564 --rest-port=41562 --tcp-port=41563 --discv5-udp-port=41565 --rest-address=0.0.0.0 --nat=extip:172.18.25.7 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=f3c258e0d8cf6ce9b9600bb1af7337dcb04cfdbf8b0ac88abdcfd28e14bb3ebf --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=41566 --metrics-logging=true --relay=true
2026-03-05 04:33:25.604 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.25.7 waku 83d78dc7b5177ed7d56a3889f48d22a5543afba2aa5d0c269b12565a811f148b
2026-03-05 04:33:25.625 ERROR [src.node.docker_mananger] Max retries reached for container 07efdebcd462. Exiting log stream.
2026-03-05 04:33:25.640 DEBUG [src.node.docker_mananger] Container started with ID 83d78dc7b517. Setting up logs at ./log/docker/node1_2026-03-05_04-33-25__92661b1f-c157-4ecb-b0d5-b480081a8057__wakuorg_nwaku:latest.log
2026-03-05 04:33:25.641 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 41562
2026-03-05 04:33:25.642 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:33:26.180 ERROR [src.node.docker_mananger] Max retries reached for container 816d9ab000ae. Exiting log stream.
2026-03-05 04:33:26.642 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:41562/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:26.646 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:33:26.646 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:33:26.646 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:41562/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:26.648 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.25.7/tcp/41563/p2p/16Uiu2HAmSyJ5Nb7Jt8gkEnF2qTNqc8VU9Faab7h5ouaDm92FeTTo","/ip4/172.18.25.7/tcp/41564/ws/p2p/16Uiu2HAmSyJ5Nb7Jt8gkEnF2qTNqc8VU9Faab7h5ouaDm92FeTTo"],"enrUri":"enr:-L24QFOfGWNkieotz4RAi27Zdh5q_dN-imEx6tRDBXUg9x36VEnkeq-RQIjb-VcEJ9n7n2VAbz4ScMeXnRp47S4EbFICgmlkgnY0gmlwhKwSGQeKbXVsdGlhZGRyc5YACASsEhkHBqJbAAoErBIZBwaiXN0DgnJzhQADAQAAiXNlY3AyNTZrMaED1LQdKItax5xQdFcdKuO5jtQFkq1h5UQ7qnrDZXfXNgKDdGNwgqJbg3VkcIKiXYV3YWt1MgE"}'
2026-03-05 04:33:26.649 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:33:26.656 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:33:26.656 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-05_04-33-25__92661b1f-c157-4ecb-b0d5-b480081a8057__wakuorg_nwaku:latest.log
2026-03-05 04:33:26.656 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:33:26.656 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:33:26.657 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:33:26.658 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.122.253
2026-03-05 04:33:26.658 DEBUG [src.node.docker_mananger] Generated ports ['5158', '5159', '5160', '5161', '5162']
2026-03-05 04:33:26.658 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:33:26.658 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:33:26.658 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:33:26.658 DEBUG [src.node.docker_mananger] docker run -i -t -p 5158:5158 -p 5159:5159 -p 5160:5160 -p 5161:5161 -p 5162:5162 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=5160 --rest-port=5158 --tcp-port=5159 --discv5-udp-port=5161 --rest-address=0.0.0.0 --nat=extip:172.18.122.253 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=df2ab66e6d4b3fbb5749dfcb6c9aebaa3da6f6d913cf076b44ce46ae27fdbfbd --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=5162 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QFOfGWNkieotz4RAi27Zdh5q_dN-imEx6tRDBXUg9x36VEnkeq-RQIjb-VcEJ9n7n2VAbz4ScMeXnRp47S4EbFICgmlkgnY0gmlwhKwSGQeKbXVsdGlhZGRyc5YACASsEhkHBqJbAAoErBIZBwaiXN0DgnJzhQADAQAAiXNlY3AyNTZrMaED1LQdKItax5xQdFcdKuO5jtQFkq1h5UQ7qnrDZXfXNgKDdGNwgqJbg3VkcIKiXYV3YWt1MgE
2026-03-05 04:33:26.851 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.122.253 waku f5cc79460cb7eb3ce88c3bbbd6523d60c8f4274615bcb4dd01649ccd02807aab
2026-03-05 04:33:26.886 DEBUG [src.node.docker_mananger] Container started with ID f5cc79460cb7. Setting up logs at ./log/docker/node2_2026-03-05_04-33-25__92661b1f-c157-4ecb-b0d5-b480081a8057__wakuorg_nwaku:latest.log
2026-03-05 04:33:26.886 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 5158
2026-03-05 04:33:26.887 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:33:27.887 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5158/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:27.909 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:33:27.911 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:33:27.912 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5158/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:27.922 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.122.253/tcp/5159/p2p/16Uiu2HAmUTHiVnAVXatvsnHk4hSmEcxKsXRG1Q4J7zVPA5Vj4aG3","/ip4/172.18.122.253/tcp/5160/ws/p2p/16Uiu2HAmUTHiVnAVXatvsnHk4hSmEcxKsXRG1Q4J7zVPA5Vj4aG3"],"enrUri":"enr:-L24QEf_cryb65HkV9JTva_0M34QU2IRVrTaBdWC3atXps9cAXtDkyblcJNUiC9NtohI8y8zVB7FIdRXKo3RyDnO-ZoCgmlkgnY0gmlwhKwSev2KbXVsdGlhZGRyc5YACASsEnr9BhQnAAoErBJ6_QYUKN0DgnJzhQADAQAAiXNlY3AyNTZrMaED6ruoEK8G_2uYFPZhPCIZNI-nvlkBeoq6tzIFzYrAzySDdGNwghQng3VkcIIUKYV3YWt1MgE"}'
2026-03-05 04:33:27.922 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:33:27.923 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:5158/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.25.7/tcp/41563/p2p/16Uiu2HAmSyJ5Nb7Jt8gkEnF2qTNqc8VU9Faab7h5ouaDm92FeTTo"]'
2026-03-05 04:33:27.927 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:27.928 DEBUG [src.steps.relay] Running fixture setup: setup_optional_relay_nodes
2026-03-05 04:33:27.935 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:33:27.935 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-05_04-33-25__92661b1f-c157-4ecb-b0d5-b480081a8057__wakuorg_nwaku:latest.log
2026-03-05 04:33:27.935 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:33:27.935 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:33:27.937 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:33:27.937 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.104.89
2026-03-05 04:33:27.937 DEBUG [src.node.docker_mananger] Generated ports ['2044', '2045', '2046', '2047', '2048']
2026-03-05 04:33:27.937 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:33:27.937 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:33:27.938 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:33:27.938 DEBUG [src.node.docker_mananger] docker run -i -t -p 2044:2044 -p 2045:2045 -p 2046:2046 -p 2047:2047 -p 2048:2048 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=2046 --rest-port=2044 --tcp-port=2045 --discv5-udp-port=2047 --rest-address=0.0.0.0 --nat=extip:172.18.104.89 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=011f4f3d14851be3e860e5fc2944fc67b3df41df7bfe1b3916c9642b7ef8dede --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=2048 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QFOfGWNkieotz4RAi27Zdh5q_dN-imEx6tRDBXUg9x36VEnkeq-RQIjb-VcEJ9n7n2VAbz4ScMeXnRp47S4EbFICgmlkgnY0gmlwhKwSGQeKbXVsdGlhZGRyc5YACASsEhkHBqJbAAoErBIZBwaiXN0DgnJzhQADAQAAiXNlY3AyNTZrMaED1LQdKItax5xQdFcdKuO5jtQFkq1h5UQ7qnrDZXfXNgKDdGNwgqJbg3VkcIKiXYV3YWt1MgE
2026-03-05 04:33:28.142 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.104.89 waku cb922606027f2ec6126ad0d8cd2567a941c97d058e461a6c1e8328c5b5596eff
2026-03-05 04:33:28.176 DEBUG [src.node.docker_mananger] Container started with ID cb922606027f. Setting up logs at ./log/docker/node3_2026-03-05_04-33-25__92661b1f-c157-4ecb-b0d5-b480081a8057__wakuorg_nwaku:latest.log
2026-03-05 04:33:28.176 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 2044
2026-03-05 04:33:28.177 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:33:29.177 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2044/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:29.190 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:33:29.191 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:33:29.191 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2044/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:29.197 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.104.89/tcp/2045/p2p/16Uiu2HAm7qQNPWSHgPgEpX1u8KNX75mMi1pdTyH3jxky8B1PvGRi","/ip4/172.18.104.89/tcp/2046/ws/p2p/16Uiu2HAm7qQNPWSHgPgEpX1u8KNX75mMi1pdTyH3jxky8B1PvGRi"],"enrUri":"enr:-L24QJvtB2lfIlaKZIaXvw24WH_M-NzqbsugRsCKV4v2ZjwNXrtG1GmS9AxWGovykFrJRVatB_oJHVumoRMeiTRVfG8CgmlkgnY0gmlwhKwSaFmKbXVsdGlhZGRyc5YACASsEmhZBgf9AAoErBJoWQYH_t0DgnJzhQADAQAAiXNlY3AyNTZrMaECuGCIIhr1JGIzg7iLHjVjlyknT97ACPOsra_3WyhpPl2DdGNwggf9g3VkcIIH_4V3YWt1MgE"}'
2026-03-05 04:33:29.198 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:33:29.199 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:2044/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.25.7/tcp/41563/p2p/16Uiu2HAmSyJ5Nb7Jt8gkEnF2qTNqc8VU9Faab7h5ouaDm92FeTTo"]'
2026-03-05 04:33:29.202 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:29.209 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:33:29.209 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-05_04-33-25__92661b1f-c157-4ecb-b0d5-b480081a8057__wakuorg_nwaku:latest.log
2026-03-05 04:33:29.209 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:33:29.209 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:33:29.210 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:33:29.211 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.12.20
2026-03-05 04:33:29.211 DEBUG [src.node.docker_mananger] Generated ports ['46903', '46904', '46905', '46906', '46907']
2026-03-05 04:33:29.211 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:33:29.211 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:33:29.211 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:33:29.211 DEBUG [src.node.docker_mananger] docker run -i -t -p 46903:46903 -p 46904:46904 -p 46905:46905 -p 46906:46906 -p 46907:46907 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=46905 --rest-port=46903 --tcp-port=46904 --discv5-udp-port=46906 --rest-address=0.0.0.0 --nat=extip:172.18.12.20 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=11fe64e307b41fecefe5ea09aa77a3f8de98cb5af51b6c4ffe8ffb4992891334 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=46907 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QFOfGWNkieotz4RAi27Zdh5q_dN-imEx6tRDBXUg9x36VEnkeq-RQIjb-VcEJ9n7n2VAbz4ScMeXnRp47S4EbFICgmlkgnY0gmlwhKwSGQeKbXVsdGlhZGRyc5YACASsEhkHBqJbAAoErBIZBwaiXN0DgnJzhQADAQAAiXNlY3AyNTZrMaED1LQdKItax5xQdFcdKuO5jtQFkq1h5UQ7qnrDZXfXNgKDdGNwgqJbg3VkcIKiXYV3YWt1MgE
2026-03-05 04:33:29.412 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.12.20 waku 9bc1ec0098a245c3955346fb48dd91968803fd938d9e1ef907ea422d16be6588
2026-03-05 04:33:29.447 DEBUG [src.node.docker_mananger] Container started with ID 9bc1ec0098a2. Setting up logs at ./log/docker/node4_2026-03-05_04-33-25__92661b1f-c157-4ecb-b0d5-b480081a8057__wakuorg_nwaku:latest.log
2026-03-05 04:33:29.448 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 46903
2026-03-05 04:33:29.448 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:33:30.450 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:46903/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:30.467 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:33:30.468 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:33:30.470 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:46903/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:30.475 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.12.20/tcp/46904/p2p/16Uiu2HAm9sZhPY3VCKX8Zno5qWjWcSRHDX6DRHDFCGwCPLDfN6LE","/ip4/172.18.12.20/tcp/46905/ws/p2p/16Uiu2HAm9sZhPY3VCKX8Zno5qWjWcSRHDX6DRHDFCGwCPLDfN6LE"],"enrUri":"enr:-L24QDya-62JGboI7PrRU8o8XK6f2iWkVEboYGPNRkVmJnV-Kg-JwxQN5aAhA_4zyfHK-uvy0JD1mWw8OLYCis2y55ICgmlkgnY0gmlwhKwSDBSKbXVsdGlhZGRyc5YACASsEgwUBrc4AAoErBIMFAa3Od0DgnJzhQADAQAAiXNlY3AyNTZrMaEC1qWdLi5neyO6fHTjReKV1yoi2YCpG51bmAcNwe9O1_eDdGNwgrc4g3VkcIK3OoV3YWt1MgE"}'
2026-03-05 04:33:30.475 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:33:30.475 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:46903/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.25.7/tcp/41563/p2p/16Uiu2HAmSyJ5Nb7Jt8gkEnF2qTNqc8VU9Faab7h5ouaDm92FeTTo"]'
2026-03-05 04:33:30.478 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:30.485 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:33:30.485 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-03-05_04-33-25__92661b1f-c157-4ecb-b0d5-b480081a8057__wakuorg_nwaku:latest.log
2026-03-05 04:33:30.485 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:33:30.485 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:33:30.486 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:33:30.486 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.127.80
2026-03-05 04:33:30.487 DEBUG [src.node.docker_mananger] Generated ports ['47499', '47500', '47501', '47502', '47503']
2026-03-05 04:33:30.487 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:33:30.487 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:33:30.487 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:33:30.487 DEBUG [src.node.docker_mananger] docker run -i -t -p 47499:47499 -p 47500:47500 -p 47501:47501 -p 47502:47502 -p 47503:47503 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=47501 --rest-port=47499 --tcp-port=47500 --discv5-udp-port=47502 --rest-address=0.0.0.0 --nat=extip:172.18.127.80 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=adbfc0984bfda54cacbb5dbb8de2e9b1aae804f6ca206e73d3dcd9e672e28dcf --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=47503 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QFOfGWNkieotz4RAi27Zdh5q_dN-imEx6tRDBXUg9x36VEnkeq-RQIjb-VcEJ9n7n2VAbz4ScMeXnRp47S4EbFICgmlkgnY0gmlwhKwSGQeKbXVsdGlhZGRyc5YACASsEhkHBqJbAAoErBIZBwaiXN0DgnJzhQADAQAAiXNlY3AyNTZrMaED1LQdKItax5xQdFcdKuO5jtQFkq1h5UQ7qnrDZXfXNgKDdGNwgqJbg3VkcIKiXYV3YWt1MgE
2026-03-05 04:33:30.693 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.127.80 waku 9e82033244cb055f4ee9a93db218ef0939935b9469e0e546adb8f722e3e9965d
2026-03-05 04:33:30.729 DEBUG [src.node.docker_mananger] Container started with ID 9e82033244cb. Setting up logs at ./log/docker/node5_2026-03-05_04-33-25__92661b1f-c157-4ecb-b0d5-b480081a8057__wakuorg_nwaku:latest.log
2026-03-05 04:33:30.730 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 47499
2026-03-05 04:33:30.730 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:33:31.730 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:47499/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:31.749 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:33:31.750 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:33:31.750 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:47499/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:31.756 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.127.80/tcp/47500/p2p/16Uiu2HAmLQGgojB7qdDqPcoYQBzjcmFzPJEQT6oRP37kaRnTwxRZ","/ip4/172.18.127.80/tcp/47501/ws/p2p/16Uiu2HAmLQGgojB7qdDqPcoYQBzjcmFzPJEQT6oRP37kaRnTwxRZ"],"enrUri":"enr:-L24QCKLokqpfgE3pMav0DH5pjH5p12T-RUSSusuU9XeSDJ_Hj8oklD4J_X3sHWCA0BNYoGrzPq9mnS8A7wE7T1YvucCgmlkgnY0gmlwhKwSf1CKbXVsdGlhZGRyc5YACASsEn9QBrmMAAoErBJ_UAa5jd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDcxhBSBISitYVprGICdRJAP8p9eSGJKBZnGxrwdJdTPyDdGNwgrmMg3VkcIK5joV3YWt1MgE"}'
2026-03-05 04:33:31.757 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:33:31.758 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:47499/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.25.7/tcp/41563/p2p/16Uiu2HAmSyJ5Nb7Jt8gkEnF2qTNqc8VU9Faab7h5ouaDm92FeTTo"]'
2026-03-05 04:33:31.761 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:31.761 DEBUG [src.steps.relay] Running fixture setup: subscribe_main_relay_nodes
2026-03-05 04:33:31.761 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:41562/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-05 04:33:31.769 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:31.769 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:5158/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-05 04:33:31.778 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:31.781 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:41562/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 04:33:31.786 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:31.786 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-05 04:33:31.887 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:31.887 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:41562/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:31.890 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772685211781636352,"ephemeral":false,"proof":""}]'
2026-03-05 04:33:31.892 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:31.892 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:5158/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:31.894 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772685211781636352,"ephemeral":false,"proof":""}]'
2026-03-05 04:33:31.896 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:41562/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 04:33:31.902 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:31.902 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-05 04:33:32.003 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:32.003 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2044/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:32.006 ERROR [src.node.api_clients.base_client] HTTP error occurred: 404 Client Error: Not Found for url: http://127.0.0.1:2044/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1. Response content: b''
2026-03-05 04:33:32.009 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-05 04:33:32.010 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-05 04:33:32.010 DEBUG [src.node.waku_node] Stopping container with id 83d78dc7b517
2026-03-05 04:33:32.605 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:33:32.609 DEBUG [src.node.waku_node] Stopping container with id f5cc79460cb7
2026-03-05 04:33:33.204 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:33:33.204 DEBUG [src.node.waku_node] Stopping container with id cb922606027f
2026-03-05 04:33:33.422 ERROR [src.node.docker_mananger] Max retries reached for container 83d78dc7b517. Exiting log stream.
2026-03-05 04:33:33.742 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:33:33.744 DEBUG [src.node.waku_node] Stopping container with id 9bc1ec0098a2
2026-03-05 04:33:34.046 ERROR [src.node.docker_mananger] Max retries reached for container f5cc79460cb7. Exiting log stream.
2026-03-05 04:33:34.306 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:33:34.307 DEBUG [src.node.waku_node] Stopping container with id 9e82033244cb
2026-03-05 04:33:34.555 ERROR [src.node.docker_mananger] Max retries reached for container cb922606027f. Exiting log stream.
2026-03-05 04:33:34.849 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:33:34.851 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-05 04:33:34.866 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:33:34.877 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:33:34.881 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:33:34.886 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:33:34.890 DEBUG [src.node.docker_mananger] No errors found in the waku logs.