2025-12-18 04:11:43.714 DEBUG [tests.conftest] Running fixture setup: test_id 2025-12-18 04:11:43.714 DEBUG [tests.conftest] Running test: test_filter_get_message_while_one_peer_is_paused with id: 2025-12-18_04-11-43__858f51e5-6338-488f-bf04-bf10c7b46a20 2025-12-18 04:11:43.715 DEBUG [src.steps.common] Running fixture setup: common_setup 2025-12-18 04:11:43.715 DEBUG [src.steps.filter] Running fixture setup: filter_setup 2025-12-18 04:11:43.715 DEBUG [src.steps.filter] Running fixture setup: setup_main_relay_node 2025-12-18 04:11:43.722 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest 2025-12-18 04:11:43.722 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2025-12-18_04-11-43__858f51e5-6338-488f-bf04-bf10c7b46a20__wakuorg_nwaku:latest.log 2025-12-18 04:11:43.722 DEBUG [src.node.waku_node] Starting Node... 2025-12-18 04:11:43.722 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku 2025-12-18 04:11:43.723 DEBUG [src.node.docker_mananger] Network waku already exists 2025-12-18 04:11:43.724 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.221.104 2025-12-18 04:11:43.724 DEBUG [src.node.docker_mananger] Generated ports ['32379', '32380', '32381', '32382', '32383'] 2025-12-18 04:11:43.724 DEBUG [src.node.waku_node] RLN credentials were not set 2025-12-18 04:11:43.724 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN 2025-12-18 04:11:43.724 DEBUG [src.node.waku_node] Using volumes [] 2025-12-18 04:11:43.724 DEBUG [src.node.docker_mananger] docker run -i -t -p 32379:32379 -p 32380:32380 -p 32381:32381 -p 32382:32382 -p 32383:32383 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=32381 --rest-port=32379 --tcp-port=32380 --discv5-udp-port=32382 --rest-address=0.0.0.0 --nat=extip:172.18.221.104 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=efb79a73d3a368cf2dfb87a68d0046798afbddaed6511bae4affb4aec0041159 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=32383 --metrics-logging=true --relay=true --filter=true 2025-12-18 04:11:43.906 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.221.104 waku 2a5ee4ef6187c3d99f84b639aa81d82e71907b30bbbf67a0f248427bfb6c0e41 2025-12-18 04:11:43.937 DEBUG [src.node.docker_mananger] Container started with ID 2a5ee4ef6187. Setting up logs at ./log/docker/node1_2025-12-18_04-11-43__858f51e5-6338-488f-bf04-bf10c7b46a20__wakuorg_nwaku:latest.log 2025-12-18 04:11:43.938 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 32379 2025-12-18 04:11:43.938 DEBUG [src.libs.common] Sleeping for 1 seconds 2025-12-18 04:11:43.949 ERROR [src.node.docker_mananger] Max retries reached for container 9295647362b5. Exiting log stream. 2025-12-18 04:11:44.497 ERROR [src.node.docker_mananger] Max retries reached for container 134472e492e1. Exiting log stream. 2025-12-18 04:11:44.938 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32379/health" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:44.942 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}' 2025-12-18 04:11:44.943 INFO [src.node.waku_node] Node protocols are initialized !! 2025-12-18 04:11:44.943 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32379/debug/v1/info" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:44.945 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.221.104/tcp/32380/p2p/16Uiu2HAmRigPqgVVMnMQWohVqfxxzfGKSP37QL3bkgJeWDLHfFXK","/ip4/172.18.221.104/tcp/32381/ws/p2p/16Uiu2HAmRigPqgVVMnMQWohVqfxxzfGKSP37QL3bkgJeWDLHfFXK"],"enrUri":"enr:-L24QPE7WSYbIV6rU8Idh7pBS9hGrTqEVMsTsTdAGDdG0NJiPS3GNhJKOhgMPRcJP9Y2nPkbQX6BBl22EvEB-LSTOc0CgmlkgnY0gmlwhKwS3WiKbXVsdGlhZGRyc5YACASsEt1oBn58AAoErBLdaAZ-fd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDwhnzSObznnsl5OnWgXW-nfL7emIFxqhxDzlIgioXeyaDdGNwgn58g3VkcIJ-foV3YWt1MgU"}' 2025-12-18 04:11:44.945 INFO [src.node.waku_node] REST service is ready !! 2025-12-18 04:11:44.946 DEBUG [src.steps.filter] Running fixture setup: setup_main_filter_node 2025-12-18 04:11:44.953 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest 2025-12-18 04:11:44.953 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2025-12-18_04-11-43__858f51e5-6338-488f-bf04-bf10c7b46a20__wakuorg_nwaku:latest.log 2025-12-18 04:11:44.953 DEBUG [src.node.waku_node] Starting Node... 2025-12-18 04:11:44.953 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku 2025-12-18 04:11:44.955 DEBUG [src.node.docker_mananger] Network waku already exists 2025-12-18 04:11:44.955 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.14.184 2025-12-18 04:11:44.955 DEBUG [src.node.docker_mananger] Generated ports ['1028', '1029', '1030', '1031', '1032'] 2025-12-18 04:11:44.955 DEBUG [src.node.waku_node] RLN credentials were not set 2025-12-18 04:11:44.956 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN 2025-12-18 04:11:44.956 DEBUG [src.node.waku_node] Using volumes [] 2025-12-18 04:11:44.956 DEBUG [src.node.docker_mananger] docker run -i -t -p 1028:1028 -p 1029:1029 -p 1030:1030 -p 1031:1031 -p 1032:1032 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=1030 --rest-port=1028 --tcp-port=1029 --discv5-udp-port=1031 --rest-address=0.0.0.0 --nat=extip:172.18.14.184 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=3a848ccfd7daf68e2bedbcdeaff27aeae38806b9faec97f5ed6ddddfd378b8ca --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=1032 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QPE7WSYbIV6rU8Idh7pBS9hGrTqEVMsTsTdAGDdG0NJiPS3GNhJKOhgMPRcJP9Y2nPkbQX6BBl22EvEB-LSTOc0CgmlkgnY0gmlwhKwS3WiKbXVsdGlhZGRyc5YACASsEt1oBn58AAoErBLdaAZ-fd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDwhnzSObznnsl5OnWgXW-nfL7emIFxqhxDzlIgioXeyaDdGNwgn58g3VkcIJ-foV3YWt1MgU --filternode=/ip4/172.18.221.104/tcp/32380/p2p/16Uiu2HAmRigPqgVVMnMQWohVqfxxzfGKSP37QL3bkgJeWDLHfFXK 2025-12-18 04:11:45.150 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.14.184 waku 869a8a96e9e89aa62cf8c80234979839867cc8a0670411c43ca013b0b03ea7b3 2025-12-18 04:11:45.179 DEBUG [src.node.docker_mananger] Container started with ID 869a8a96e9e8. Setting up logs at ./log/docker/node2_2025-12-18_04-11-43__858f51e5-6338-488f-bf04-bf10c7b46a20__wakuorg_nwaku:latest.log 2025-12-18 04:11:45.180 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 1028 2025-12-18 04:11:45.180 DEBUG [src.libs.common] Sleeping for 1 seconds 2025-12-18 04:11:46.180 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1028/health" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:46.184 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"}]}' 2025-12-18 04:11:46.185 INFO [src.node.waku_node] Node protocols are initialized !! 2025-12-18 04:11:46.185 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1028/debug/v1/info" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:46.187 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.14.184/tcp/1029/p2p/16Uiu2HAmNaViqVfF2C1UrMqXC9CPfwxpkizh7oVPtRzKXYvPUcqs","/ip4/172.18.14.184/tcp/1030/ws/p2p/16Uiu2HAmNaViqVfF2C1UrMqXC9CPfwxpkizh7oVPtRzKXYvPUcqs"],"enrUri":"enr:-L24QMbV86QHq-StbZNGeXDGf0Q0fIV3zfst4cMo3GQusZiZJsT9O1-DN2qTXMO2vnIHIHsXb5QzPc68N47VzPoBEKcCgmlkgnY0gmlwhKwSDriKbXVsdGlhZGRyc5YACASsEg64BgQFAAoErBIOuAYEBt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDk24s-VzVX3OFZgVBW-dG1hdgGaEZ_taPJptSKWpuA3aDdGNwggQFg3VkcIIEB4V3YWt1MgA"}' 2025-12-18 04:11:46.187 INFO [src.node.waku_node] REST service is ready !! 2025-12-18 04:11:46.188 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:1028/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.221.104/tcp/32380/p2p/16Uiu2HAmRigPqgVVMnMQWohVqfxxzfGKSP37QL3bkgJeWDLHfFXK"]' 2025-12-18 04:11:46.220 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK' 2025-12-18 04:11:46.231 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest 2025-12-18 04:11:46.232 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2025-12-18_04-11-43__858f51e5-6338-488f-bf04-bf10c7b46a20__wakuorg_nwaku:latest.log 2025-12-18 04:11:46.233 DEBUG [src.node.waku_node] Starting Node... 2025-12-18 04:11:46.233 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku 2025-12-18 04:11:46.238 DEBUG [src.node.docker_mananger] Network waku already exists 2025-12-18 04:11:46.238 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.151.189 2025-12-18 04:11:46.239 DEBUG [src.node.docker_mananger] Generated ports ['42807', '42808', '42809', '42810', '42811'] 2025-12-18 04:11:46.239 DEBUG [src.node.waku_node] RLN credentials were not set 2025-12-18 04:11:46.240 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN 2025-12-18 04:11:46.240 DEBUG [src.node.waku_node] Using volumes [] 2025-12-18 04:11:46.240 DEBUG [src.node.docker_mananger] docker run -i -t -p 42807:42807 -p 42808:42808 -p 42809:42809 -p 42810:42810 -p 42811:42811 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=42809 --rest-port=42807 --tcp-port=42808 --discv5-udp-port=42810 --rest-address=0.0.0.0 --nat=extip:172.18.151.189 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9c47b0a18ab926be2440984c3dfc24e86c1dae310bf8739d92146eade3cc7fea --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=42811 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QPE7WSYbIV6rU8Idh7pBS9hGrTqEVMsTsTdAGDdG0NJiPS3GNhJKOhgMPRcJP9Y2nPkbQX6BBl22EvEB-LSTOc0CgmlkgnY0gmlwhKwS3WiKbXVsdGlhZGRyc5YACASsEt1oBn58AAoErBLdaAZ-fd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDwhnzSObznnsl5OnWgXW-nfL7emIFxqhxDzlIgioXeyaDdGNwgn58g3VkcIJ-foV3YWt1MgU --filternode=/ip4/172.18.221.104/tcp/32380/p2p/16Uiu2HAmRigPqgVVMnMQWohVqfxxzfGKSP37QL3bkgJeWDLHfFXK 2025-12-18 04:11:46.438 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.151.189 waku 27cba76e11972ae57e23d10e1e1e1ca79056eed2ac6e54703b8861f7d3c50c5b 2025-12-18 04:11:46.468 DEBUG [src.node.docker_mananger] Container started with ID 27cba76e1197. Setting up logs at ./log/docker/node3_2025-12-18_04-11-43__858f51e5-6338-488f-bf04-bf10c7b46a20__wakuorg_nwaku:latest.log 2025-12-18 04:11:46.468 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 42807 2025-12-18 04:11:46.468 DEBUG [src.libs.common] Sleeping for 1 seconds 2025-12-18 04:11:47.469 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42807/health" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:47.473 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"}]}' 2025-12-18 04:11:47.473 INFO [src.node.waku_node] Node protocols are initialized !! 2025-12-18 04:11:47.473 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42807/debug/v1/info" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:47.476 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.151.189/tcp/42808/p2p/16Uiu2HAmDdmUydoF2WnvtsKnw2fWHVwdqhYE4UyJ6Y9sXzV7icYi","/ip4/172.18.151.189/tcp/42809/ws/p2p/16Uiu2HAmDdmUydoF2WnvtsKnw2fWHVwdqhYE4UyJ6Y9sXzV7icYi"],"enrUri":"enr:-L24QJIIY0mUY9EVPsUUn7eNh4TCM9DsIQZrraiHkJlAu97yEGDV8pT6gpsBDP666srW5wE2Zz8KtVQgIMgwu6Og8hICgmlkgnY0gmlwhKwSl72KbXVsdGlhZGRyc5YACASsEpe9Bqc4AAoErBKXvQanOd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDDouOjMMLBfHXWr9jtuMRscAYNLxYPmEnRx-qWZCGhKODdGNwgqc4g3VkcIKnOoV3YWt1MgA"}' 2025-12-18 04:11:47.476 INFO [src.node.waku_node] REST service is ready !! 2025-12-18 04:11:47.476 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:42807/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.221.104/tcp/32380/p2p/16Uiu2HAmRigPqgVVMnMQWohVqfxxzfGKSP37QL3bkgJeWDLHfFXK"]' 2025-12-18 04:11:47.504 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK' 2025-12-18 04:11:47.516 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest 2025-12-18 04:11:47.517 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2025-12-18_04-11-43__858f51e5-6338-488f-bf04-bf10c7b46a20__wakuorg_nwaku:latest.log 2025-12-18 04:11:47.517 DEBUG [src.node.waku_node] Starting Node... 2025-12-18 04:11:47.517 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku 2025-12-18 04:11:47.519 DEBUG [src.node.docker_mananger] Network waku already exists 2025-12-18 04:11:47.519 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.5.176 2025-12-18 04:11:47.519 DEBUG [src.node.docker_mananger] Generated ports ['30819', '30820', '30821', '30822', '30823'] 2025-12-18 04:11:47.519 DEBUG [src.node.waku_node] RLN credentials were not set 2025-12-18 04:11:47.519 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN 2025-12-18 04:11:47.519 DEBUG [src.node.waku_node] Using volumes [] 2025-12-18 04:11:47.520 DEBUG [src.node.docker_mananger] docker run -i -t -p 30819:30819 -p 30820:30820 -p 30821:30821 -p 30822:30822 -p 30823:30823 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=30821 --rest-port=30819 --tcp-port=30820 --discv5-udp-port=30822 --rest-address=0.0.0.0 --nat=extip:172.18.5.176 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=84b82b09b2bfdacce46d5f5698bbcdcdc6d7c2b1b3fdfca8cc1b51dfa8d4ccbd --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=30823 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QPE7WSYbIV6rU8Idh7pBS9hGrTqEVMsTsTdAGDdG0NJiPS3GNhJKOhgMPRcJP9Y2nPkbQX6BBl22EvEB-LSTOc0CgmlkgnY0gmlwhKwS3WiKbXVsdGlhZGRyc5YACASsEt1oBn58AAoErBLdaAZ-fd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDwhnzSObznnsl5OnWgXW-nfL7emIFxqhxDzlIgioXeyaDdGNwgn58g3VkcIJ-foV3YWt1MgU --filternode=/ip4/172.18.221.104/tcp/32380/p2p/16Uiu2HAmRigPqgVVMnMQWohVqfxxzfGKSP37QL3bkgJeWDLHfFXK 2025-12-18 04:11:47.707 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.5.176 waku 5fb91c2078a75f683265cfcc28eb828fe5fa545fd23642e7f03ee96855050fbf 2025-12-18 04:11:47.739 DEBUG [src.node.docker_mananger] Container started with ID 5fb91c2078a7. Setting up logs at ./log/docker/node4_2025-12-18_04-11-43__858f51e5-6338-488f-bf04-bf10c7b46a20__wakuorg_nwaku:latest.log 2025-12-18 04:11:47.740 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 30819 2025-12-18 04:11:47.740 DEBUG [src.libs.common] Sleeping for 1 seconds 2025-12-18 04:11:48.741 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30819/health" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:48.745 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"}]}' 2025-12-18 04:11:48.745 INFO [src.node.waku_node] Node protocols are initialized !! 2025-12-18 04:11:48.746 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30819/debug/v1/info" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:48.748 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.5.176/tcp/30820/p2p/16Uiu2HAmPQZ8PuDhBsbpReWyjKv4giUPhvAa1iyYf8rSXyie3u1i","/ip4/172.18.5.176/tcp/30821/ws/p2p/16Uiu2HAmPQZ8PuDhBsbpReWyjKv4giUPhvAa1iyYf8rSXyie3u1i"],"enrUri":"enr:-L24QKjON19rc7JaFVRkCNXygLBIbj1jTSeMKRdG4PGl2Fl_N4SwaYQYDZXU4kCzOoitnN5csP17NWCvsHUm1wP07ZoCgmlkgnY0gmlwhKwSBbCKbXVsdGlhZGRyc5YACASsEgWwBnhkAAoErBIFsAZ4Zd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDn73n6SPW5F_VaKzI_JgntgsFw-9oNbKwF4IVHjb9QbmDdGNwgnhkg3VkcIJ4ZoV3YWt1MgA"}' 2025-12-18 04:11:48.748 INFO [src.node.waku_node] REST service is ready !! 2025-12-18 04:11:48.749 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30819/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.221.104/tcp/32380/p2p/16Uiu2HAmRigPqgVVMnMQWohVqfxxzfGKSP37QL3bkgJeWDLHfFXK"]' 2025-12-18 04:11:48.776 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK' 2025-12-18 04:11:48.788 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest 2025-12-18 04:11:48.789 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2025-12-18_04-11-43__858f51e5-6338-488f-bf04-bf10c7b46a20__wakuorg_nwaku:latest.log 2025-12-18 04:11:48.790 DEBUG [src.node.waku_node] Starting Node... 2025-12-18 04:11:48.790 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku 2025-12-18 04:11:48.792 DEBUG [src.node.docker_mananger] Network waku already exists 2025-12-18 04:11:48.793 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.55.118 2025-12-18 04:11:48.793 DEBUG [src.node.docker_mananger] Generated ports ['23280', '23281', '23282', '23283', '23284'] 2025-12-18 04:11:48.793 DEBUG [src.node.waku_node] RLN credentials were not set 2025-12-18 04:11:48.793 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN 2025-12-18 04:11:48.793 DEBUG [src.node.waku_node] Using volumes [] 2025-12-18 04:11:48.793 DEBUG [src.node.docker_mananger] docker run -i -t -p 23280:23280 -p 23281:23281 -p 23282:23282 -p 23283:23283 -p 23284:23284 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=23282 --rest-port=23280 --tcp-port=23281 --discv5-udp-port=23283 --rest-address=0.0.0.0 --nat=extip:172.18.55.118 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=de4ecdafcf937cdf96e6ebbaa4fcff0a704f3ed34e4e8c3e00d4d6754752f04a --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=23284 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QPE7WSYbIV6rU8Idh7pBS9hGrTqEVMsTsTdAGDdG0NJiPS3GNhJKOhgMPRcJP9Y2nPkbQX6BBl22EvEB-LSTOc0CgmlkgnY0gmlwhKwS3WiKbXVsdGlhZGRyc5YACASsEt1oBn58AAoErBLdaAZ-fd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDwhnzSObznnsl5OnWgXW-nfL7emIFxqhxDzlIgioXeyaDdGNwgn58g3VkcIJ-foV3YWt1MgU --filternode=/ip4/172.18.221.104/tcp/32380/p2p/16Uiu2HAmRigPqgVVMnMQWohVqfxxzfGKSP37QL3bkgJeWDLHfFXK 2025-12-18 04:11:48.992 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.55.118 waku fb6fdaadd5a53cb35a18545f0e8552e3282c8f710eb27bc46eb4f81de2308ec2 2025-12-18 04:11:49.023 DEBUG [src.node.docker_mananger] Container started with ID fb6fdaadd5a5. Setting up logs at ./log/docker/node5_2025-12-18_04-11-43__858f51e5-6338-488f-bf04-bf10c7b46a20__wakuorg_nwaku:latest.log 2025-12-18 04:11:49.023 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 23280 2025-12-18 04:11:49.023 DEBUG [src.libs.common] Sleeping for 1 seconds 2025-12-18 04:11:50.024 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23280/health" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:50.028 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"}]}' 2025-12-18 04:11:50.028 INFO [src.node.waku_node] Node protocols are initialized !! 2025-12-18 04:11:50.028 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23280/debug/v1/info" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:50.030 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.55.118/tcp/23281/p2p/16Uiu2HAm4KTcsBcavv6y3bFdz56bqsjx8M6WciEZw8ReC36xLoR2","/ip4/172.18.55.118/tcp/23282/ws/p2p/16Uiu2HAm4KTcsBcavv6y3bFdz56bqsjx8M6WciEZw8ReC36xLoR2"],"enrUri":"enr:-L24QCDGL9H4uUyX13z1UdUTkBPz8GpUr6UJSHh3QT4M-sWpe7bPO9bSE7tARU0XnLomHWcufP5Bpod-nS1mor0tIJQCgmlkgnY0gmlwhKwSN3aKbXVsdGlhZGRyc5YACASsEjd2BlrxAAoErBI3dgZa8t0DgnJzhQADAQAAiXNlY3AyNTZrMaEChCG4l22vZ4lrbkB8QlbmqA6ym59yhaUTvJ3fY5CiTnGDdGNwglrxg3VkcIJa84V3YWt1MgA"}' 2025-12-18 04:11:50.031 INFO [src.node.waku_node] REST service is ready !! 2025-12-18 04:11:50.031 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23280/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.221.104/tcp/32380/p2p/16Uiu2HAmRigPqgVVMnMQWohVqfxxzfGKSP37QL3bkgJeWDLHfFXK"]' 2025-12-18 04:11:50.065 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK' 2025-12-18 04:11:50.067 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32379/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]' 2025-12-18 04:11:50.073 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK' 2025-12-18 04:11:50.074 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:1028/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "91ce2a48-a425-42e8-8453-aada7cd25074", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}' 2025-12-18 04:11:50.085 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"91ce2a48-a425-42e8-8453-aada7cd25074","statusDesc":"OK"}' 2025-12-18 04:11:50.086 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:42807/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "07846da7-f68f-437a-ab6f-b0d869d367c9", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}' 2025-12-18 04:11:50.095 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"07846da7-f68f-437a-ab6f-b0d869d367c9","statusDesc":"OK"}' 2025-12-18 04:11:50.095 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30819/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "47bcb08b-b482-4f44-8672-4bf32545b7a6", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}' 2025-12-18 04:11:50.106 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"47bcb08b-b482-4f44-8672-4bf32545b7a6","statusDesc":"OK"}' 2025-12-18 04:11:50.106 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23280/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "f3dd01c4-d431-4dfc-afa0-8a6f47dc1f3b", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}' 2025-12-18 04:11:50.115 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"f3dd01c4-d431-4dfc-afa0-8a6f47dc1f3b","statusDesc":"OK"}' 2025-12-18 04:11:50.116 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32379/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "RmlsdGVyIHdvcmtzISE=", "contentTopic": "/test/1/waku-filter/proto", "timestamp": '$(date +%s%N)'}' 2025-12-18 04:11:50.132 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK' 2025-12-18 04:11:50.134 DEBUG [src.libs.common] Sleeping for 0.1 seconds 2025-12-18 04:11:50.234 DEBUG [src.steps.filter] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message 2025-12-18 04:11:50.235 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1028/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:50.238 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1766031110116403481,"ephemeral":false}]' 2025-12-18 04:11:50.240 DEBUG [src.steps.filter] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message 2025-12-18 04:11:50.240 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42807/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:50.243 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1766031110116403481,"ephemeral":false}]' 2025-12-18 04:11:50.244 DEBUG [src.steps.filter] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message 2025-12-18 04:11:50.244 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30819/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:50.247 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1766031110116403481,"ephemeral":false}]' 2025-12-18 04:11:50.248 DEBUG [src.steps.filter] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message 2025-12-18 04:11:50.249 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23280/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:50.251 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1766031110116403481,"ephemeral":false}]' 2025-12-18 04:11:50.253 DEBUG [src.node.waku_node] Pausing container with id 869a8a96e9e8 2025-12-18 04:11:50.262 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32379/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "RmlsdGVyIHdvcmtzISE=", "contentTopic": "/test/1/waku-filter/proto", "timestamp": '$(date +%s%N)'}' 2025-12-18 04:11:50.268 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK' 2025-12-18 04:11:50.269 DEBUG [src.node.waku_node] Unpause container with id 869a8a96e9e8 2025-12-18 04:11:50.279 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32379/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "RmlsdGVyIHdvcmtzISE=", "contentTopic": "/test/1/waku-filter/proto", "timestamp": '$(date +%s%N)'}' 2025-12-18 04:11:50.286 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK' 2025-12-18 04:11:50.286 DEBUG [src.libs.common] Sleeping for 0.5 seconds 2025-12-18 04:11:50.787 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1028/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None' 2025-12-18 04:11:50.790 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1766031110253052447,"ephemeral":false},{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1766031110253129901,"ephemeral":false}]' 2025-12-18 04:11:50.791 DEBUG [tests.conftest] Running fixture teardown: test_setup 2025-12-18 04:11:50.792 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes 2025-12-18 04:11:50.792 DEBUG [src.node.waku_node] Stopping container with id 2a5ee4ef6187 2025-12-18 04:11:51.358 DEBUG [src.node.waku_node] Container stopped. 2025-12-18 04:11:51.358 DEBUG [src.node.waku_node] Stopping container with id 869a8a96e9e8 2025-12-18 04:11:51.919 DEBUG [src.node.waku_node] Container stopped. 2025-12-18 04:11:51.919 DEBUG [src.node.waku_node] Stopping container with id 27cba76e1197 2025-12-18 04:11:52.172 ERROR [src.node.docker_mananger] Max retries reached for container 2a5ee4ef6187. Exiting log stream. 2025-12-18 04:11:52.471 DEBUG [src.node.waku_node] Container stopped. 2025-12-18 04:11:52.472 DEBUG [src.node.waku_node] Stopping container with id 5fb91c2078a7 2025-12-18 04:11:52.731 ERROR [src.node.docker_mananger] Max retries reached for container 869a8a96e9e8. Exiting log stream. 2025-12-18 04:11:53.018 DEBUG [src.node.waku_node] Container stopped. 2025-12-18 04:11:53.018 DEBUG [src.node.waku_node] Stopping container with id fb6fdaadd5a5 2025-12-18 04:11:53.285 ERROR [src.node.docker_mananger] Max retries reached for container 27cba76e1197. Exiting log stream. 2025-12-18 04:11:53.519 DEBUG [src.node.waku_node] Container stopped. 2025-12-18 04:11:53.521 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors 2025-12-18 04:11:53.535 DEBUG [src.node.docker_mananger] No errors found in the waku logs. 2025-12-18 04:11:53.540 DEBUG [src.node.docker_mananger] No errors found in the waku logs. 2025-12-18 04:11:53.544 DEBUG [src.node.docker_mananger] No errors found in the waku logs. 2025-12-18 04:11:53.549 DEBUG [src.node.docker_mananger] No errors found in the waku logs. 2025-12-18 04:11:53.553 DEBUG [src.node.docker_mananger] No errors found in the waku logs.