201 lines
42 KiB
Plaintext

2026-03-26 04:46:01.100 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-26 04:46:01.100 DEBUG [tests.conftest] Running test: test_store_filter_interaction_with_six_nodes with id: 2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882
2026-03-26 04:46:01.100 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-26 04:46:01.101 DEBUG [src.steps.filter] Running fixture setup: filter_setup
2026-03-26 04:46:01.101 DEBUG [src.steps.light_push] Running fixture setup: light_push_setup
2026-03-26 04:46:01.111 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-26 04:46:01.111 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882__wakuorg_nwaku:latest.log
2026-03-26 04:46:01.119 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-26 04:46:01.119 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882__wakuorg_nwaku:latest.log
2026-03-26 04:46:01.126 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-26 04:46:01.126 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882__wakuorg_nwaku:latest.log
2026-03-26 04:46:01.126 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-26 04:46:01.126 DEBUG [src.steps.store] Running fixture setup: store_setup
2026-03-26 04:46:01.127 DEBUG [tests.e2e.test_e2e] Create 6 nodes
2026-03-26 04:46:01.133 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-26 04:46:01.133 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882__wakuorg_nwaku:latest.log
2026-03-26 04:46:01.139 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-26 04:46:01.139 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882__wakuorg_nwaku:latest.log
2026-03-26 04:46:01.144 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-26 04:46:01.144 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node6_2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882__wakuorg_nwaku:latest.log
2026-03-26 04:46:01.145 DEBUG [tests.e2e.test_e2e] Start 5 nodes with their corresponding config
2026-03-26 04:46:01.145 DEBUG [src.node.waku_node] Starting Node...
2026-03-26 04:46:01.145 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-26 04:46:01.188 DEBUG [src.node.docker_mananger] Network waku created
2026-03-26 04:46:01.189 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.197.250
2026-03-26 04:46:01.189 DEBUG [src.node.docker_mananger] Generated ports ['59946', '59947', '59948', '59949', '59950']
2026-03-26 04:46:01.189 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-26 04:46:01.189 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-26 04:46:01.189 DEBUG [src.node.waku_node] Using volumes []
2026-03-26 04:46:01.189 DEBUG [src.node.docker_mananger] docker run -i -t -p 59946:59946 -p 59947:59947 -p 59948:59948 -p 59949:59949 -p 59950:59950 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=59948 --rest-port=59946 --tcp-port=59947 --discv5-udp-port=59949 --rest-address=0.0.0.0 --nat=extip:172.18.197.250 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=bcabfa3ebea29f83408fd2bcb67e812ddf64cfc12c8f2ac13ba75a14fdcedb8d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=59950 --metrics-logging=true --relay=true --store=true
2026-03-26 04:46:04.275 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.197.250 waku da5818b14c86cbd5c796c1c82742a2e41d5bda429f541c36ba2081adb81ad25d
2026-03-26 04:46:04.312 DEBUG [src.node.docker_mananger] Container started with ID da5818b14c86. Setting up logs at ./log/docker/node1_2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882__wakuorg_nwaku:latest.log
2026-03-26 04:46:04.312 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 59946
2026-03-26 04:46:04.313 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-26 04:46:05.313 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59946/health" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:05.316 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-26 04:46:05.317 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-26 04:46:05.317 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59946/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:05.319 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.197.250/tcp/59947/p2p/16Uiu2HAm6nZj2QzDYvTt8gzujJpP58kEb43uCfmjfiyzfvnXoXS3","/ip4/172.18.197.250/tcp/59948/ws/p2p/16Uiu2HAm6nZj2QzDYvTt8gzujJpP58kEb43uCfmjfiyzfvnXoXS3"],"enrUri":"enr:-L24QMS-08sq0fdolWqW3fnYHiPbPWzvA-Ar9YyUTldzKjL7WfiqLMclhUK4aNC4XQ2SN46PJe_1E7cxwJ8Q9i37H3kCgmlkgnY0gmlwhKwSxfqKbXVsdGlhZGRyc5YACASsEsX6BuorAAoErBLF-gbqLN0DgnJzhQADAQAAiXNlY3AyNTZrMaECqMquWWp_lWRJdJDN8qbiFownZkzy80tyIo1U_Qs9T1SDdGNwguorg3VkcILqLYV3YWt1MgM"}'
2026-03-26 04:46:05.319 INFO [src.node.waku_node] REST service is ready !!
2026-03-26 04:46:05.319 DEBUG [src.node.waku_node] Starting Node...
2026-03-26 04:46:05.320 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-26 04:46:05.321 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-26 04:46:05.321 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.108.220
2026-03-26 04:46:05.321 DEBUG [src.node.docker_mananger] Generated ports ['37997', '37998', '37999', '38000', '38001']
2026-03-26 04:46:05.322 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-26 04:46:05.322 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-26 04:46:05.322 DEBUG [src.node.waku_node] Using volumes []
2026-03-26 04:46:05.322 DEBUG [src.node.docker_mananger] docker run -i -t -p 37997:37997 -p 37998:37998 -p 37999:37999 -p 38000:38000 -p 38001:38001 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=37999 --rest-port=37997 --tcp-port=37998 --discv5-udp-port=38000 --rest-address=0.0.0.0 --nat=extip:172.18.108.220 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=fa2ecd2e1ceeafb9ccb6f2f76bbadfe14bad4cddcb4bd248935b5a0e8fcac932 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=38001 --metrics-logging=true --relay=true --store=true --discv5-bootstrap-node=enr:-L24QMS-08sq0fdolWqW3fnYHiPbPWzvA-Ar9YyUTldzKjL7WfiqLMclhUK4aNC4XQ2SN46PJe_1E7cxwJ8Q9i37H3kCgmlkgnY0gmlwhKwSxfqKbXVsdGlhZGRyc5YACASsEsX6BuorAAoErBLF-gbqLN0DgnJzhQADAQAAiXNlY3AyNTZrMaECqMquWWp_lWRJdJDN8qbiFownZkzy80tyIo1U_Qs9T1SDdGNwguorg3VkcILqLYV3YWt1MgM
2026-03-26 04:46:05.524 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.108.220 waku 278164b936d15a3468c942b1b8fc6f55161103a7c246442b9f9ff0e645723452
2026-03-26 04:46:05.559 DEBUG [src.node.docker_mananger] Container started with ID 278164b936d1. Setting up logs at ./log/docker/node2_2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882__wakuorg_nwaku:latest.log
2026-03-26 04:46:05.560 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 37997
2026-03-26 04:46:05.560 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-26 04:46:06.561 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:37997/health" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:06.565 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-26 04:46:06.565 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-26 04:46:06.565 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:37997/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:06.568 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.108.220/tcp/37998/p2p/16Uiu2HAmKXKo7KjCNUT6wuBvbofGWFh9xEZ5Wezp2SF6YCciUGVS","/ip4/172.18.108.220/tcp/37999/ws/p2p/16Uiu2HAmKXKo7KjCNUT6wuBvbofGWFh9xEZ5Wezp2SF6YCciUGVS"],"enrUri":"enr:-L24QEB2MTZfvUhkCp4GqKk4ahhvtB2c9ZoFf30mfpuNSnbAbGOq228gtJluZ7TEQlVr_ZuyhfzZ5Hh98M0AxgEIB8YCgmlkgnY0gmlwhKwSbNyKbXVsdGlhZGRyc5YACASsEmzcBpRuAAoErBJs3AaUb90DgnJzhQADAQAAiXNlY3AyNTZrMaEDZgsmE6nslfssJZ6DZFFbHRAItzmI4auCyLeyVBpD0XWDdGNwgpRug3VkcIKUcIV3YWt1MgM"}'
2026-03-26 04:46:06.568 INFO [src.node.waku_node] REST service is ready !!
2026-03-26 04:46:06.568 DEBUG [src.node.waku_node] Starting Node...
2026-03-26 04:46:06.568 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-26 04:46:06.570 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-26 04:46:06.570 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.5.246
2026-03-26 04:46:06.570 DEBUG [src.node.docker_mananger] Generated ports ['11826', '11827', '11828', '11829', '11830']
2026-03-26 04:46:06.570 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-26 04:46:06.571 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-26 04:46:06.571 DEBUG [src.node.waku_node] Using volumes []
2026-03-26 04:46:06.571 DEBUG [src.node.docker_mananger] docker run -i -t -p 11826:11826 -p 11827:11827 -p 11828:11828 -p 11829:11829 -p 11830:11830 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=11828 --rest-port=11826 --tcp-port=11827 --discv5-udp-port=11829 --rest-address=0.0.0.0 --nat=extip:172.18.5.246 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=867c1edab49b5e0d9c7b4e14aaf05b4babdaf9dcda1bedcc7b308b5d1d2d8111 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=11830 --metrics-logging=true --relay=true --store=true --discv5-bootstrap-node=enr:-L24QEB2MTZfvUhkCp4GqKk4ahhvtB2c9ZoFf30mfpuNSnbAbGOq228gtJluZ7TEQlVr_ZuyhfzZ5Hh98M0AxgEIB8YCgmlkgnY0gmlwhKwSbNyKbXVsdGlhZGRyc5YACASsEmzcBpRuAAoErBJs3AaUb90DgnJzhQADAQAAiXNlY3AyNTZrMaEDZgsmE6nslfssJZ6DZFFbHRAItzmI4auCyLeyVBpD0XWDdGNwgpRug3VkcIKUcIV3YWt1MgM
2026-03-26 04:46:06.801 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.5.246 waku 3b985ec97c65e02c9d1e8eac3df96b6915aada9ca2e917e4d18f2e75553f581a
2026-03-26 04:46:06.838 DEBUG [src.node.docker_mananger] Container started with ID 3b985ec97c65. Setting up logs at ./log/docker/node3_2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882__wakuorg_nwaku:latest.log
2026-03-26 04:46:06.838 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 11826
2026-03-26 04:46:06.838 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-26 04:46:07.839 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:11826/health" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:07.842 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-26 04:46:07.842 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-26 04:46:07.842 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:11826/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:07.844 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.5.246/tcp/11827/p2p/16Uiu2HAm2hLrncinHeNs3miYCT8LMRK7K8wAk8mBh94oiFofNnBQ","/ip4/172.18.5.246/tcp/11828/ws/p2p/16Uiu2HAm2hLrncinHeNs3miYCT8LMRK7K8wAk8mBh94oiFofNnBQ"],"enrUri":"enr:-L24QHv4G_CDCbiBEo5safxE2EBAXPt50P0_SpHS9TEdpBynMWf6nnt7taR8J4ThaafsAkIeN24jqCMMiftMOtGw3cgCgmlkgnY0gmlwhKwSBfaKbXVsdGlhZGRyc5YACASsEgX2Bi4zAAoErBIF9gYuNN0DgnJzhQADAQAAiXNlY3AyNTZrMaECbAV7Qf2eGhePE8ZcrxTM4EQaRuslhuGubVnh7cP3rBeDdGNwgi4zg3VkcIIuNYV3YWt1MgM"}'
2026-03-26 04:46:07.845 INFO [src.node.waku_node] REST service is ready !!
2026-03-26 04:46:07.845 DEBUG [src.node.waku_node] Starting Node...
2026-03-26 04:46:07.845 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-26 04:46:07.847 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-26 04:46:07.847 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.68.115
2026-03-26 04:46:07.847 DEBUG [src.node.docker_mananger] Generated ports ['36607', '36608', '36609', '36610', '36611']
2026-03-26 04:46:07.847 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-26 04:46:07.847 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-26 04:46:07.847 DEBUG [src.node.waku_node] Using volumes []
2026-03-26 04:46:07.847 DEBUG [src.node.docker_mananger] docker run -i -t -p 36607:36607 -p 36608:36608 -p 36609:36609 -p 36610:36610 -p 36611:36611 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=36609 --rest-port=36607 --tcp-port=36608 --discv5-udp-port=36610 --rest-address=0.0.0.0 --nat=extip:172.18.68.115 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=85a044b9845ecfb145aa1968ece30ade0bd7d5be43c351cf8e90cc1d2edb8c0c --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=36611 --metrics-logging=true --relay=true --filter=true --store=true --discv5-bootstrap-node=enr:-L24QHv4G_CDCbiBEo5safxE2EBAXPt50P0_SpHS9TEdpBynMWf6nnt7taR8J4ThaafsAkIeN24jqCMMiftMOtGw3cgCgmlkgnY0gmlwhKwSBfaKbXVsdGlhZGRyc5YACASsEgX2Bi4zAAoErBIF9gYuNN0DgnJzhQADAQAAiXNlY3AyNTZrMaECbAV7Qf2eGhePE8ZcrxTM4EQaRuslhuGubVnh7cP3rBeDdGNwgi4zg3VkcIIuNYV3YWt1MgM
2026-03-26 04:46:08.075 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.68.115 waku c0f4f33d8bb88a9f089d3b0ed4a43a7b2a991e824908b02137c9dd8fae13e44d
2026-03-26 04:46:08.111 DEBUG [src.node.docker_mananger] Container started with ID c0f4f33d8bb8. Setting up logs at ./log/docker/node4_2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882__wakuorg_nwaku:latest.log
2026-03-26 04:46:08.111 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 36607
2026-03-26 04:46:08.111 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-26 04:46:09.113 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36607/health" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:09.116 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-26 04:46:09.116 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-26 04:46:09.117 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36607/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:09.119 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.68.115/tcp/36608/p2p/16Uiu2HAmN4hYSyv2jmBbk7faSEh78gQjWM9p3pBj4HLANMLL6S6D","/ip4/172.18.68.115/tcp/36609/ws/p2p/16Uiu2HAmN4hYSyv2jmBbk7faSEh78gQjWM9p3pBj4HLANMLL6S6D"],"enrUri":"enr:-L24QFrgPHjKCTlI0uIZS8NPAwyE3Jh_j8dtRQ6Ld8_hV-WlCgZH-KOCZM--zN3dK1KL3Gs8JXEuiPzcQiLLBxYlXrQCgmlkgnY0gmlwhKwSRHOKbXVsdGlhZGRyc5YACASsEkRzBo8AAAoErBJEcwaPAd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDi8weCawlSU-AVpjTW9JmfYcXtcJIBBIcWmTnf136CAqDdGNwgo8Ag3VkcIKPAoV3YWt1Mgc"}'
2026-03-26 04:46:09.119 INFO [src.node.waku_node] REST service is ready !!
2026-03-26 04:46:09.119 DEBUG [src.node.waku_node] Starting Node...
2026-03-26 04:46:09.119 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-26 04:46:09.121 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-26 04:46:09.121 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.35.139
2026-03-26 04:46:09.121 DEBUG [src.node.docker_mananger] Generated ports ['10220', '10221', '10222', '10223', '10224']
2026-03-26 04:46:09.121 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-26 04:46:09.122 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-26 04:46:09.122 DEBUG [src.node.waku_node] Using volumes []
2026-03-26 04:46:09.122 DEBUG [src.node.docker_mananger] docker run -i -t -p 10220:10220 -p 10221:10221 -p 10222:10222 -p 10223:10223 -p 10224:10224 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=10222 --rest-port=10220 --tcp-port=10221 --discv5-udp-port=10223 --rest-address=0.0.0.0 --nat=extip:172.18.35.139 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=fb2b291afcb0b8a1bb2fdc1c659a7a61c0d4c1fa0a4af608b8ed48af28afe9fc --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=10224 --metrics-logging=true --relay=false --filternode=/ip4/172.18.68.115/tcp/36608/p2p/16Uiu2HAmN4hYSyv2jmBbk7faSEh78gQjWM9p3pBj4HLANMLL6S6D --discv5-bootstrap-node=enr:-L24QFrgPHjKCTlI0uIZS8NPAwyE3Jh_j8dtRQ6Ld8_hV-WlCgZH-KOCZM--zN3dK1KL3Gs8JXEuiPzcQiLLBxYlXrQCgmlkgnY0gmlwhKwSRHOKbXVsdGlhZGRyc5YACASsEkRzBo8AAAoErBJEcwaPAd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDi8weCawlSU-AVpjTW9JmfYcXtcJIBBIcWmTnf136CAqDdGNwgo8Ag3VkcIKPAoV3YWt1Mgc
2026-03-26 04:46:09.349 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.35.139 waku 27d4a301e3d34cf45cf395e7de4d49d5d68101203ced86aa5bf4c45fd3d1be5a
2026-03-26 04:46:09.381 DEBUG [src.node.docker_mananger] Container started with ID 27d4a301e3d3. Setting up logs at ./log/docker/node6_2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882__wakuorg_nwaku:latest.log
2026-03-26 04:46:09.381 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 10220
2026-03-26 04:46:09.383 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-26 04:46:10.385 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10220/health" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:10.394 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-26 04:46:10.395 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-26 04:46:10.396 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10220/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:10.399 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.35.139/tcp/10221/p2p/16Uiu2HAmKTy5NuaqDxDqaCKtUbJpoXP6E8rXQrSZV6wXQ4hLfPgC","/ip4/172.18.35.139/tcp/10222/ws/p2p/16Uiu2HAmKTy5NuaqDxDqaCKtUbJpoXP6E8rXQrSZV6wXQ4hLfPgC"],"enrUri":"enr:-L24QP3FMtNK1t36BXCuwUdsSYBKckxXkUiHoHnf-t1ZyYAudPbqgSP9TfDQbXMUnyqql0xOOpWs9QfUcFIPwGouzK4CgmlkgnY0gmlwhKwSI4uKbXVsdGlhZGRyc5YACASsEiOLBiftAAoErBIjiwYn7t0DgnJzhQADAQAAiXNlY3AyNTZrMaEDZS76i3uUPN6sN6t44y8-52ReM560nrfp-tUtBJRK11mDdGNwgiftg3VkcIIn74V3YWt1MgA"}'
2026-03-26 04:46:10.400 INFO [src.node.waku_node] REST service is ready !!
2026-03-26 04:46:10.400 DEBUG [tests.e2e.test_e2e] Subscribe nodes to relay pubsub topics
2026-03-26 04:46:10.402 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:59946/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-26 04:46:10.415 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-26 04:46:10.415 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:37997/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-26 04:46:10.426 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-26 04:46:10.426 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:11826/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-26 04:46:10.439 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-26 04:46:10.440 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:36607/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-26 04:46:10.453 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-26 04:46:10.453 DEBUG [tests.e2e.test_e2e] Node6 subscribe to filter for pubsubtopic /waku/2/rs/3/1
2026-03-26 04:46:10.454 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:10220/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "1", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-26 04:46:10.504 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"1","statusDesc":"OK"}'
2026-03-26 04:46:10.504 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59946/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:10.513 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.5.246/tcp/11827/p2p/16Uiu2HAm2hLrncinHeNs3miYCT8LMRK7K8wAk8mBh94oiFofNnBQ","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-12-g0b8609","origin":"Discv5"},{"multiaddr":"/ip4/172.18.68.115/tcp/36608/p2p/16Uiu2HAmN4hYSyv2jmBbk7faSEh78gQjWM9p3pBj4HLANMLL6S6D","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-subscribe/2.0.0-beta1","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-12-g0b8609","origin":"Discv5"},{"multiaddr":"/ip4/172.18.108.220/tcp/56624/p2p/16Uiu2HAmKXKo7KjCNUT6wuBvbofGWFh9xEZ5Wezp2SF6YCciUGVS","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-12-g0b8609","origin":"Discv5"}]'
2026-03-26 04:46:10.514 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:37997/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:10.521 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.5.246/tcp/43384/p2p/16Uiu2HAm2hLrncinHeNs3miYCT8LMRK7K8wAk8mBh94oiFofNnBQ","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-12-g0b8609","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.197.250/tcp/59947/p2p/16Uiu2HAm6nZj2QzDYvTt8gzujJpP58kEb43uCfmjfiyzfvnXoXS3","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-12-g0b8609","origin":"Discv5"}]'
2026-03-26 04:46:10.521 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:11826/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:10.527 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.68.115/tcp/40700/p2p/16Uiu2HAmN4hYSyv2jmBbk7faSEh78gQjWM9p3pBj4HLANMLL6S6D","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-subscribe/2.0.0-beta1","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-12-g0b8609","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.197.250/tcp/33030/p2p/16Uiu2HAm6nZj2QzDYvTt8gzujJpP58kEb43uCfmjfiyzfvnXoXS3","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-12-g0b8609","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.108.220/tcp/37998/p2p/16Uiu2HAmKXKo7KjCNUT6wuBvbofGWFh9xEZ5Wezp2SF6YCciUGVS","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-12-g0b8609","origin":"Discv5"}]'
2026-03-26 04:46:10.527 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36607/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:10.530 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.5.246/tcp/11827/p2p/16Uiu2HAm2hLrncinHeNs3miYCT8LMRK7K8wAk8mBh94oiFofNnBQ","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-12-g0b8609","origin":"Discv5"},{"multiaddr":"/ip4/172.18.197.250/tcp/46166/p2p/16Uiu2HAm6nZj2QzDYvTt8gzujJpP58kEb43uCfmjfiyzfvnXoXS3","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-12-g0b8609","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.35.139/tcp/45270/p2p/16Uiu2HAmKTy5NuaqDxDqaCKtUbJpoXP6E8rXQrSZV6wXQ4hLfPgC","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-12-g0b8609","origin":"UnknownOrigin"}]'
2026-03-26 04:46:10.530 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10220/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:10.533 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.68.115/tcp/36608/p2p/16Uiu2HAmN4hYSyv2jmBbk7faSEh78gQjWM9p3pBj4HLANMLL6S6D","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-subscribe/2.0.0-beta1","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0,1],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-12-g0b8609","origin":"Discv5"}]'
2026-03-26 04:46:10.533 DEBUG [src.libs.common] Sleeping for 50 seconds
2026-03-26 04:47:00.533 DEBUG [tests.e2e.test_e2e] Node1 publish message for topic /waku/2/rs/3/1
2026-03-26 04:47:00.534 DEBUG [src.steps.store] Relaying message
2026-03-26 04:47:00.534 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:59946/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "RmlsdGVyIHdvcmtzISE=", "contentTopic": "/test/1/waku-filter/proto", "timestamp": '$(date +%s%N)'}'
2026-03-26 04:47:00.549 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-26 04:47:00.550 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-26 04:47:00.752 DEBUG [src.libs.common] Sleeping for 4 seconds
2026-03-26 04:47:04.752 DEBUG [tests.e2e.test_e2e] Node6 inquery for filter messages on pubsubtopic /waku/2/rs/3/1 & contenttopic/test/1/waku-filter/proto
2026-03-26 04:47:04.753 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10220/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:47:04.756 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1774500420534124586,"ephemeral":false}]'
2026-03-26 04:47:04.756 DEBUG [tests.e2e.test_e2e] Filter inquiry response is [{'payload': 'RmlsdGVyIHdvcmtzISE=', 'contentTopic': '/test/1/waku-filter/proto', 'version': 0, 'timestamp': 1774500420534124586, 'ephemeral': False}]
2026-03-26 04:47:04.756 DEBUG [tests.e2e.test_e2e] Node5 goes live !!
2026-03-26 04:47:04.756 DEBUG [src.node.waku_node] Starting Node...
2026-03-26 04:47:04.756 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-26 04:47:04.759 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-26 04:47:04.759 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.17.124
2026-03-26 04:47:04.759 DEBUG [src.node.docker_mananger] Generated ports ['46973', '46974', '46975', '46976', '46977']
2026-03-26 04:47:04.760 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-26 04:47:04.760 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-26 04:47:04.760 DEBUG [src.node.waku_node] Using volumes []
2026-03-26 04:47:04.760 DEBUG [src.node.docker_mananger] docker run -i -t -p 46973:46973 -p 46974:46974 -p 46975:46975 -p 46976:46976 -p 46977:46977 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=46975 --rest-port=46973 --tcp-port=46974 --discv5-udp-port=46976 --rest-address=0.0.0.0 --nat=extip:172.18.17.124 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=dbf3e8c799db3bdf4bac5227fb278f2df6bb5f242b0f9bbc14aa15f52682a1f5 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=46977 --metrics-logging=true --relay=false --storenode=/ip4/172.18.68.115/tcp/36608/p2p/16Uiu2HAmN4hYSyv2jmBbk7faSEh78gQjWM9p3pBj4HLANMLL6S6D --discv5-bootstrap-node=enr:-L24QFrgPHjKCTlI0uIZS8NPAwyE3Jh_j8dtRQ6Ld8_hV-WlCgZH-KOCZM--zN3dK1KL3Gs8JXEuiPzcQiLLBxYlXrQCgmlkgnY0gmlwhKwSRHOKbXVsdGlhZGRyc5YACASsEkRzBo8AAAoErBJEcwaPAd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDi8weCawlSU-AVpjTW9JmfYcXtcJIBBIcWmTnf136CAqDdGNwgo8Ag3VkcIKPAoV3YWt1Mgc
2026-03-26 04:47:04.963 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.17.124 waku 551746852fade76857536e6ae77708de30926f99bb1ab9d8d1ebece38d0c1eb2
2026-03-26 04:47:04.995 DEBUG [src.node.docker_mananger] Container started with ID 551746852fad. Setting up logs at ./log/docker/node5_2026-03-26_04-46-01__fe12b4f3-26e5-49bb-b3d1-7ea79596d882__wakuorg_nwaku:latest.log
2026-03-26 04:47:04.995 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 46973
2026-03-26 04:47:04.996 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-26 04:47:05.996 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:46973/health" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:47:05.999 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-26 04:47:05.999 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-26 04:47:05.999 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:46973/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:47:06.001 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.17.124/tcp/46974/p2p/16Uiu2HAmCJTcf94itXktmqmGLzWbom5mN1hyG7yPqVcM1ByAmCMW","/ip4/172.18.17.124/tcp/46975/ws/p2p/16Uiu2HAmCJTcf94itXktmqmGLzWbom5mN1hyG7yPqVcM1ByAmCMW"],"enrUri":"enr:-L24QJjLG6WZJQJUeJImiZfdyvLboTqkYm49Dhsf5Sj3W-0lYlf8ci1lRvlLLTzutm8HBn0tU1ReIbE6f7fa3iJSm2YCgmlkgnY0gmlwhKwSEXyKbXVsdGlhZGRyc5YACASsEhF8Brd-AAoErBIRfAa3f90DgnJzhQADAQAAiXNlY3AyNTZrMaEC-r2hsXcuqPDhS42-UmW0_3yuKrIqUjuEsxigKUkq9SGDdGNwgrd-g3VkcIK3gIV3YWt1MgA"}'
2026-03-26 04:47:06.001 INFO [src.node.waku_node] REST service is ready !!
2026-03-26 04:47:06.001 DEBUG [src.libs.common] Sleeping for 2 seconds
2026-03-26 04:47:08.002 DEBUG [tests.e2e.test_e2e] Node5 makes request to get stored messages
2026-03-26 04:47:08.002 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-26 04:47:08.003 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:46973/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F1&pageSize=50&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:47:08.049 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"8946762aff2487afd564","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0xd1b8290d038d29630feb0936994d21edee46049b04cab4e6b5bb31587947ec0b"}]}'
2026-03-26 04:47:08.051 DEBUG [src.steps.store] messages length is 1
2026-03-26 04:47:08.060 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-26 04:47:08.060 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-26 04:47:08.061 DEBUG [src.node.waku_node] Stopping container with id da5818b14c86
2026-03-26 04:47:08.649 DEBUG [src.node.waku_node] Container stopped.
2026-03-26 04:47:08.651 DEBUG [src.node.waku_node] Stopping container with id 278164b936d1
2026-03-26 04:47:09.247 DEBUG [src.node.waku_node] Container stopped.
2026-03-26 04:47:09.249 DEBUG [src.node.waku_node] Stopping container with id 3b985ec97c65
2026-03-26 04:47:09.473 ERROR [src.node.docker_mananger] Max retries reached for container da5818b14c86. Exiting log stream.
2026-03-26 04:47:09.781 DEBUG [src.node.waku_node] Container stopped.
2026-03-26 04:47:09.785 DEBUG [src.node.waku_node] Stopping container with id c0f4f33d8bb8
2026-03-26 04:47:10.331 ERROR [src.node.docker_mananger] Max retries reached for container 278164b936d1. Exiting log stream.
2026-03-26 04:47:10.334 DEBUG [src.node.waku_node] Container stopped.
2026-03-26 04:47:10.336 DEBUG [src.node.waku_node] Stopping container with id 27d4a301e3d3
2026-03-26 04:47:10.598 ERROR [src.node.docker_mananger] Max retries reached for container 3b985ec97c65. Exiting log stream.
2026-03-26 04:47:10.864 DEBUG [src.node.waku_node] Container stopped.
2026-03-26 04:47:10.865 DEBUG [src.node.waku_node] Stopping container with id 551746852fad
2026-03-26 04:47:11.391 ERROR [src.node.docker_mananger] Max retries reached for container c0f4f33d8bb8. Exiting log stream.
2026-03-26 04:47:11.406 DEBUG [src.node.waku_node] Container stopped.
2026-03-26 04:47:11.408 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-26 04:47:11.458 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-26 04:47:11.563 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-26 04:47:11.614 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-26 04:47:11.748 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-26 04:47:11.765 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-26 04:47:11.769 DEBUG [src.node.docker_mananger] No errors found in the waku logs.