177 lines
34 KiB
Plaintext

2026-03-15 04:53:31.130 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-15 04:53:31.132 DEBUG [tests.conftest] Running test: test_filter_get_message_while_one_peer_is_paused with id: 2026-03-15_04-53-31__172a0ce4-ff6b-4dbe-8fec-23e041bf4877
2026-03-15 04:53:31.132 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-15 04:53:31.132 DEBUG [src.steps.filter] Running fixture setup: filter_setup
2026-03-15 04:53:31.133 DEBUG [src.steps.filter] Running fixture setup: setup_main_relay_node
2026-03-15 04:53:31.142 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-15 04:53:31.143 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-15_04-53-31__172a0ce4-ff6b-4dbe-8fec-23e041bf4877__wakuorg_nwaku:latest.log
2026-03-15 04:53:31.143 DEBUG [src.node.waku_node] Starting Node...
2026-03-15 04:53:31.143 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-15 04:53:31.146 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-15 04:53:31.146 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.194.166
2026-03-15 04:53:31.146 DEBUG [src.node.docker_mananger] Generated ports ['51124', '51125', '51126', '51127', '51128']
2026-03-15 04:53:31.146 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-15 04:53:31.147 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-15 04:53:31.147 DEBUG [src.node.waku_node] Using volumes []
2026-03-15 04:53:31.147 DEBUG [src.node.docker_mananger] docker run -i -t -p 51124:51124 -p 51125:51125 -p 51126:51126 -p 51127:51127 -p 51128:51128 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=51126 --rest-port=51124 --tcp-port=51125 --discv5-udp-port=51127 --rest-address=0.0.0.0 --nat=extip:172.18.194.166 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=af7ef07643f5a7ffa96780cfd6e1c96380d9aafac6cdca3bc0c2d8bee61eece8 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=51128 --metrics-logging=true --relay=true --filter=true
2026-03-15 04:53:31.321 ERROR [src.node.docker_mananger] Max retries reached for container 009bb2bac58b. Exiting log stream.
2026-03-15 04:53:31.400 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.194.166 waku bb5177ea91a7945e9fcbd124595233820772cad4fd94cb7ad9eb753742d0d6b7
2026-03-15 04:53:31.448 DEBUG [src.node.docker_mananger] Container started with ID bb5177ea91a7. Setting up logs at ./log/docker/node1_2026-03-15_04-53-31__172a0ce4-ff6b-4dbe-8fec-23e041bf4877__wakuorg_nwaku:latest.log
2026-03-15 04:53:31.449 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 51124
2026-03-15 04:53:31.449 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-15 04:53:32.307 ERROR [src.node.docker_mananger] Max retries reached for container 4b43aadbae4e. Exiting log stream.
2026-03-15 04:53:32.450 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:51124/health" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:32.453 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-15 04:53:32.453 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-15 04:53:32.453 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:51124/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:32.456 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.194.166/tcp/51125/p2p/16Uiu2HAmHAiYr6Bt1JmA5zzEnygqM6sZB7iH4GzsU7FdufnX3WT4","/ip4/172.18.194.166/tcp/51126/ws/p2p/16Uiu2HAmHAiYr6Bt1JmA5zzEnygqM6sZB7iH4GzsU7FdufnX3WT4"],"enrUri":"enr:-L24QNeouP6n2E55MERd6Nna9lEtQPFRvlEFDuD0wj4VreHjWbSvD4lnvqfiVNBlvr3GDzRerxW1gDpxk8bE723iJwYCgmlkgnY0gmlwhKwSwqaKbXVsdGlhZGRyc5YACASsEsKmBse1AAoErBLCpgbHtt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDQwxOV-G8Y4mvgykqnO0F7O_0q2E0Uvada3LsYsCR28uDdGNwgse1g3VkcILHt4V3YWt1MgU"}'
2026-03-15 04:53:32.456 INFO [src.node.waku_node] REST service is ready !!
2026-03-15 04:53:32.456 DEBUG [src.steps.filter] Running fixture setup: setup_main_filter_node
2026-03-15 04:53:32.463 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-15 04:53:32.463 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-15_04-53-31__172a0ce4-ff6b-4dbe-8fec-23e041bf4877__wakuorg_nwaku:latest.log
2026-03-15 04:53:32.463 DEBUG [src.node.waku_node] Starting Node...
2026-03-15 04:53:32.463 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-15 04:53:32.465 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-15 04:53:32.465 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.87.234
2026-03-15 04:53:32.465 DEBUG [src.node.docker_mananger] Generated ports ['8438', '8439', '8440', '8441', '8442']
2026-03-15 04:53:32.465 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-15 04:53:32.465 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-15 04:53:32.465 DEBUG [src.node.waku_node] Using volumes []
2026-03-15 04:53:32.466 DEBUG [src.node.docker_mananger] docker run -i -t -p 8438:8438 -p 8439:8439 -p 8440:8440 -p 8441:8441 -p 8442:8442 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=8440 --rest-port=8438 --tcp-port=8439 --discv5-udp-port=8441 --rest-address=0.0.0.0 --nat=extip:172.18.87.234 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=0efc62ec12cbc2d64fcaf8bd5bbd5f4efba02bd4a593bae5fdae04d032f77f9b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=8442 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QNeouP6n2E55MERd6Nna9lEtQPFRvlEFDuD0wj4VreHjWbSvD4lnvqfiVNBlvr3GDzRerxW1gDpxk8bE723iJwYCgmlkgnY0gmlwhKwSwqaKbXVsdGlhZGRyc5YACASsEsKmBse1AAoErBLCpgbHtt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDQwxOV-G8Y4mvgykqnO0F7O_0q2E0Uvada3LsYsCR28uDdGNwgse1g3VkcILHt4V3YWt1MgU --filternode=/ip4/172.18.194.166/tcp/51125/p2p/16Uiu2HAmHAiYr6Bt1JmA5zzEnygqM6sZB7iH4GzsU7FdufnX3WT4
2026-03-15 04:53:32.675 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.87.234 waku 713e91819eff09b90e1529f3e12a75bd5ab09cd4a21ca19a3cf2a765804eb561
2026-03-15 04:53:32.714 DEBUG [src.node.docker_mananger] Container started with ID 713e91819eff. Setting up logs at ./log/docker/node2_2026-03-15_04-53-31__172a0ce4-ff6b-4dbe-8fec-23e041bf4877__wakuorg_nwaku:latest.log
2026-03-15 04:53:32.716 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 8438
2026-03-15 04:53:32.716 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-15 04:53:33.716 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8438/health" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:33.719 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-15 04:53:33.720 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-15 04:53:33.720 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8438/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:33.722 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.87.234/tcp/8439/p2p/16Uiu2HAmDqcMkwhxYMuBLmazDqwFiXgCSXkaxEuPjW7UKyetXTDc","/ip4/172.18.87.234/tcp/8440/ws/p2p/16Uiu2HAmDqcMkwhxYMuBLmazDqwFiXgCSXkaxEuPjW7UKyetXTDc"],"enrUri":"enr:-L24QFEP_v0A9mTM7FEOkKoZGkzzc5Q2nGH-DkUgjWh-VHHdCmvxl5vvi6r2tb5uQdMOu3TZ0P4ruK-b1BJ1aiku2B0CgmlkgnY0gmlwhKwSV-qKbXVsdGlhZGRyc5YACASsElfqBiD3AAoErBJX6gYg-N0DgnJzhQADAQAAiXNlY3AyNTZrMaEDEZQ1oJnYKbz1WYTfFuMmotj6BkUXlOkk-w74SUjN2YODdGNwgiD3g3VkcIIg-YV3YWt1MgA"}'
2026-03-15 04:53:33.723 INFO [src.node.waku_node] REST service is ready !!
2026-03-15 04:53:33.723 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:8438/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.194.166/tcp/51125/p2p/16Uiu2HAmHAiYr6Bt1JmA5zzEnygqM6sZB7iH4GzsU7FdufnX3WT4"]'
2026-03-15 04:53:33.757 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-15 04:53:33.772 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-15 04:53:33.772 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-15_04-53-31__172a0ce4-ff6b-4dbe-8fec-23e041bf4877__wakuorg_nwaku:latest.log
2026-03-15 04:53:33.772 DEBUG [src.node.waku_node] Starting Node...
2026-03-15 04:53:33.772 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-15 04:53:33.775 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-15 04:53:33.775 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.45.255
2026-03-15 04:53:33.775 DEBUG [src.node.docker_mananger] Generated ports ['27814', '27815', '27816', '27817', '27818']
2026-03-15 04:53:33.776 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-15 04:53:33.776 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-15 04:53:33.776 DEBUG [src.node.waku_node] Using volumes []
2026-03-15 04:53:33.776 DEBUG [src.node.docker_mananger] docker run -i -t -p 27814:27814 -p 27815:27815 -p 27816:27816 -p 27817:27817 -p 27818:27818 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=27816 --rest-port=27814 --tcp-port=27815 --discv5-udp-port=27817 --rest-address=0.0.0.0 --nat=extip:172.18.45.255 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=cbfed11a04bfecee9baffbd0eecc1068e5fef837f441b15a6c0cff688f77eae5 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=27818 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QNeouP6n2E55MERd6Nna9lEtQPFRvlEFDuD0wj4VreHjWbSvD4lnvqfiVNBlvr3GDzRerxW1gDpxk8bE723iJwYCgmlkgnY0gmlwhKwSwqaKbXVsdGlhZGRyc5YACASsEsKmBse1AAoErBLCpgbHtt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDQwxOV-G8Y4mvgykqnO0F7O_0q2E0Uvada3LsYsCR28uDdGNwgse1g3VkcILHt4V3YWt1MgU --filternode=/ip4/172.18.194.166/tcp/51125/p2p/16Uiu2HAmHAiYr6Bt1JmA5zzEnygqM6sZB7iH4GzsU7FdufnX3WT4
2026-03-15 04:53:33.993 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.45.255 waku fbdd2e28e9f20d5a0c5c5ed3afb73f2edda928d9a7d9acc1b150181c3d634f2e
2026-03-15 04:53:34.027 DEBUG [src.node.docker_mananger] Container started with ID fbdd2e28e9f2. Setting up logs at ./log/docker/node3_2026-03-15_04-53-31__172a0ce4-ff6b-4dbe-8fec-23e041bf4877__wakuorg_nwaku:latest.log
2026-03-15 04:53:34.028 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 27814
2026-03-15 04:53:34.028 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-15 04:53:35.028 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27814/health" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:35.032 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-15 04:53:35.032 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-15 04:53:35.032 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27814/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:35.035 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.45.255/tcp/27815/p2p/16Uiu2HAm1JePSzqkx3UDtMm6PaYqKJ6y6P2MWQz4xYGs2CYNTef9","/ip4/172.18.45.255/tcp/27816/ws/p2p/16Uiu2HAm1JePSzqkx3UDtMm6PaYqKJ6y6P2MWQz4xYGs2CYNTef9"],"enrUri":"enr:-L24QJcdovRY90qeuqjOFN59i3Td6ZktBBAopn0Oy44StkheDHMvsKb3t2-JMjS71HhlNIzeBVvgIdoqwtSKiMe_7x4CgmlkgnY0gmlwhKwSLf-KbXVsdGlhZGRyc5YACASsEi3_BmynAAoErBIt_wZsqN0DgnJzhQADAQAAiXNlY3AyNTZrMaECV1lBcjtOb7zKlUE_MxzwL1Ai8aNGdi0VzN38DwyOctiDdGNwgmyng3VkcIJsqYV3YWt1MgA"}'
2026-03-15 04:53:35.035 INFO [src.node.waku_node] REST service is ready !!
2026-03-15 04:53:35.035 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:27814/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.194.166/tcp/51125/p2p/16Uiu2HAmHAiYr6Bt1JmA5zzEnygqM6sZB7iH4GzsU7FdufnX3WT4"]'
2026-03-15 04:53:35.067 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-15 04:53:35.083 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-15 04:53:35.083 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-15_04-53-31__172a0ce4-ff6b-4dbe-8fec-23e041bf4877__wakuorg_nwaku:latest.log
2026-03-15 04:53:35.085 DEBUG [src.node.waku_node] Starting Node...
2026-03-15 04:53:35.089 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-15 04:53:35.091 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-15 04:53:35.091 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.92.31
2026-03-15 04:53:35.091 DEBUG [src.node.docker_mananger] Generated ports ['39131', '39132', '39133', '39134', '39135']
2026-03-15 04:53:35.092 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-15 04:53:35.092 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-15 04:53:35.092 DEBUG [src.node.waku_node] Using volumes []
2026-03-15 04:53:35.092 DEBUG [src.node.docker_mananger] docker run -i -t -p 39131:39131 -p 39132:39132 -p 39133:39133 -p 39134:39134 -p 39135:39135 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=39133 --rest-port=39131 --tcp-port=39132 --discv5-udp-port=39134 --rest-address=0.0.0.0 --nat=extip:172.18.92.31 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=1e25a990f0804ba783d6df7c285bb7f109b8eccb1cf963bcb52abff842fc73fa --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=39135 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QNeouP6n2E55MERd6Nna9lEtQPFRvlEFDuD0wj4VreHjWbSvD4lnvqfiVNBlvr3GDzRerxW1gDpxk8bE723iJwYCgmlkgnY0gmlwhKwSwqaKbXVsdGlhZGRyc5YACASsEsKmBse1AAoErBLCpgbHtt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDQwxOV-G8Y4mvgykqnO0F7O_0q2E0Uvada3LsYsCR28uDdGNwgse1g3VkcILHt4V3YWt1MgU --filternode=/ip4/172.18.194.166/tcp/51125/p2p/16Uiu2HAmHAiYr6Bt1JmA5zzEnygqM6sZB7iH4GzsU7FdufnX3WT4
2026-03-15 04:53:35.301 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.92.31 waku b067c870a0f8e210f5a4db1ec5b611d8d7e066772494f9c84a1b18d0e9a8075b
2026-03-15 04:53:35.339 DEBUG [src.node.docker_mananger] Container started with ID b067c870a0f8. Setting up logs at ./log/docker/node4_2026-03-15_04-53-31__172a0ce4-ff6b-4dbe-8fec-23e041bf4877__wakuorg_nwaku:latest.log
2026-03-15 04:53:35.340 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 39131
2026-03-15 04:53:35.340 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-15 04:53:36.341 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39131/health" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:36.344 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-15 04:53:36.344 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-15 04:53:36.344 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39131/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:36.347 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.92.31/tcp/39132/p2p/16Uiu2HAmQPdkjeGLTKCD5FKGMgx9DqCR8HDPWce7oCyk4F9vdNVn","/ip4/172.18.92.31/tcp/39133/ws/p2p/16Uiu2HAmQPdkjeGLTKCD5FKGMgx9DqCR8HDPWce7oCyk4F9vdNVn"],"enrUri":"enr:-L24QFUs8IedovK2_mS2nTfJ9_VOlN-27DiC4n6Cu1-Xa-0Gb3KAOPGQoMH45cxLMEg1BXsIZWIWiZp1aPEku6CT3g8CgmlkgnY0gmlwhKwSXB-KbXVsdGlhZGRyc5YACASsElwfBpjcAAoErBJcHwaY3d0DgnJzhQADAQAAiXNlY3AyNTZrMaEDrl0-R-atKo-fUHvKsM0yKpFVxP0seWjjt4Jw52-yl2mDdGNwgpjcg3VkcIKY3oV3YWt1MgA"}'
2026-03-15 04:53:36.347 INFO [src.node.waku_node] REST service is ready !!
2026-03-15 04:53:36.347 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39131/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.194.166/tcp/51125/p2p/16Uiu2HAmHAiYr6Bt1JmA5zzEnygqM6sZB7iH4GzsU7FdufnX3WT4"]'
2026-03-15 04:53:36.384 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-15 04:53:36.397 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-15 04:53:36.398 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-03-15_04-53-31__172a0ce4-ff6b-4dbe-8fec-23e041bf4877__wakuorg_nwaku:latest.log
2026-03-15 04:53:36.398 DEBUG [src.node.waku_node] Starting Node...
2026-03-15 04:53:36.398 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-15 04:53:36.401 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-15 04:53:36.401 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.80.67
2026-03-15 04:53:36.402 DEBUG [src.node.docker_mananger] Generated ports ['23117', '23118', '23119', '23120', '23121']
2026-03-15 04:53:36.403 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-15 04:53:36.403 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-15 04:53:36.403 DEBUG [src.node.waku_node] Using volumes []
2026-03-15 04:53:36.403 DEBUG [src.node.docker_mananger] docker run -i -t -p 23117:23117 -p 23118:23118 -p 23119:23119 -p 23120:23120 -p 23121:23121 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=23119 --rest-port=23117 --tcp-port=23118 --discv5-udp-port=23120 --rest-address=0.0.0.0 --nat=extip:172.18.80.67 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=a93fcbbd3bf1997f5253328ceb8ba3436a1a27ec8feb006292d053fb0fea2aa5 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=23121 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QNeouP6n2E55MERd6Nna9lEtQPFRvlEFDuD0wj4VreHjWbSvD4lnvqfiVNBlvr3GDzRerxW1gDpxk8bE723iJwYCgmlkgnY0gmlwhKwSwqaKbXVsdGlhZGRyc5YACASsEsKmBse1AAoErBLCpgbHtt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDQwxOV-G8Y4mvgykqnO0F7O_0q2E0Uvada3LsYsCR28uDdGNwgse1g3VkcILHt4V3YWt1MgU --filternode=/ip4/172.18.194.166/tcp/51125/p2p/16Uiu2HAmHAiYr6Bt1JmA5zzEnygqM6sZB7iH4GzsU7FdufnX3WT4
2026-03-15 04:53:36.623 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.80.67 waku f57acd93d7e826634659ce2fdae197dc27b41a69991f5b13ab0a51dd2546eac5
2026-03-15 04:53:36.660 DEBUG [src.node.docker_mananger] Container started with ID f57acd93d7e8. Setting up logs at ./log/docker/node5_2026-03-15_04-53-31__172a0ce4-ff6b-4dbe-8fec-23e041bf4877__wakuorg_nwaku:latest.log
2026-03-15 04:53:36.661 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 23117
2026-03-15 04:53:36.661 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-15 04:53:37.662 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23117/health" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:37.665 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-15 04:53:37.666 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-15 04:53:37.666 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23117/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:37.668 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.80.67/tcp/23118/p2p/16Uiu2HAm9Gm9gvuSmioxLwMe8pgpz2yLY9KFxNp8cHH8YMiW1PRL","/ip4/172.18.80.67/tcp/23119/ws/p2p/16Uiu2HAm9Gm9gvuSmioxLwMe8pgpz2yLY9KFxNp8cHH8YMiW1PRL"],"enrUri":"enr:-L24QIdLlcHNSXqNqVq8qrWWKd1B-vrhE5q9rDiaBLk0j1gOaviiQ-YABN-z0BEj_rJTJyybjTnNwOON57cym9zIdxMCgmlkgnY0gmlwhKwSUEOKbXVsdGlhZGRyc5YACASsElBDBlpOAAoErBJQQwZaT90DgnJzhQADAQAAiXNlY3AyNTZrMaECzbs8cnbO6eLAdmCUWF1IqnpvthZ18ed5L5Q7p_k3SYuDdGNwglpOg3VkcIJaUIV3YWt1MgA"}'
2026-03-15 04:53:37.668 INFO [src.node.waku_node] REST service is ready !!
2026-03-15 04:53:37.669 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23117/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.194.166/tcp/51125/p2p/16Uiu2HAmHAiYr6Bt1JmA5zzEnygqM6sZB7iH4GzsU7FdufnX3WT4"]'
2026-03-15 04:53:37.706 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-15 04:53:37.709 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:51124/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-15 04:53:37.719 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-15 04:53:37.720 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:8438/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "8fce80a2-0aac-4525-97ee-c7c8b0ea1347", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-15 04:53:37.730 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"8fce80a2-0aac-4525-97ee-c7c8b0ea1347","statusDesc":"OK"}'
2026-03-15 04:53:37.731 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:27814/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "b072d0c1-5ce2-4772-9a0d-b9b87ede2f05", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-15 04:53:37.741 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"b072d0c1-5ce2-4772-9a0d-b9b87ede2f05","statusDesc":"OK"}'
2026-03-15 04:53:37.742 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39131/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "462f106e-cae1-4121-85c2-6140470b6ded", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-15 04:53:37.752 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"462f106e-cae1-4121-85c2-6140470b6ded","statusDesc":"OK"}'
2026-03-15 04:53:37.753 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23117/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "e32aa3af-3c88-48bb-82e2-694fb760f374", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-15 04:53:37.764 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"e32aa3af-3c88-48bb-82e2-694fb760f374","statusDesc":"OK"}'
2026-03-15 04:53:37.765 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:51124/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "RmlsdGVyIHdvcmtzISE=", "contentTopic": "/test/1/waku-filter/proto", "timestamp": '$(date +%s%N)'}'
2026-03-15 04:53:37.783 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-15 04:53:37.784 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-15 04:53:37.884 DEBUG [src.steps.filter] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-15 04:53:37.885 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8438/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:37.888 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1773550417765280034,"ephemeral":false}]'
2026-03-15 04:53:37.890 DEBUG [src.steps.filter] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-03-15 04:53:37.890 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27814/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:37.893 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1773550417765280034,"ephemeral":false}]'
2026-03-15 04:53:37.894 DEBUG [src.steps.filter] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-03-15 04:53:37.895 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39131/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:37.897 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1773550417765280034,"ephemeral":false}]'
2026-03-15 04:53:37.899 DEBUG [src.steps.filter] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-03-15 04:53:37.899 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23117/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:37.902 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1773550417765280034,"ephemeral":false}]'
2026-03-15 04:53:37.903 DEBUG [src.node.waku_node] Pausing container with id 713e91819eff
2026-03-15 04:53:37.914 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:51124/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "RmlsdGVyIHdvcmtzISE=", "contentTopic": "/test/1/waku-filter/proto", "timestamp": '$(date +%s%N)'}'
2026-03-15 04:53:37.921 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-15 04:53:37.921 DEBUG [src.node.waku_node] Unpause container with id 713e91819eff
2026-03-15 04:53:37.931 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:51124/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "RmlsdGVyIHdvcmtzISE=", "contentTopic": "/test/1/waku-filter/proto", "timestamp": '$(date +%s%N)'}'
2026-03-15 04:53:37.938 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-15 04:53:37.939 DEBUG [src.libs.common] Sleeping for 0.5 seconds
2026-03-15 04:53:38.440 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8438/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-15 04:53:38.443 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1773550417903843628,"ephemeral":false},{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1773550417903921133,"ephemeral":false}]'
2026-03-15 04:53:38.445 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-15 04:53:38.446 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-15 04:53:38.446 DEBUG [src.node.waku_node] Stopping container with id bb5177ea91a7
2026-03-15 04:53:39.014 DEBUG [src.node.waku_node] Container stopped.
2026-03-15 04:53:39.015 DEBUG [src.node.waku_node] Stopping container with id 713e91819eff
2026-03-15 04:53:39.657 DEBUG [src.node.waku_node] Container stopped.
2026-03-15 04:53:39.660 DEBUG [src.node.waku_node] Stopping container with id fbdd2e28e9f2
2026-03-15 04:53:39.830 ERROR [src.node.docker_mananger] Max retries reached for container bb5177ea91a7. Exiting log stream.
2026-03-15 04:53:40.235 DEBUG [src.node.waku_node] Container stopped.
2026-03-15 04:53:40.236 DEBUG [src.node.waku_node] Stopping container with id b067c870a0f8
2026-03-15 04:53:40.473 ERROR [src.node.docker_mananger] Max retries reached for container 713e91819eff. Exiting log stream.
2026-03-15 04:53:40.804 DEBUG [src.node.waku_node] Container stopped.
2026-03-15 04:53:40.805 DEBUG [src.node.waku_node] Stopping container with id f57acd93d7e8
2026-03-15 04:53:41.074 ERROR [src.node.docker_mananger] Max retries reached for container fbdd2e28e9f2. Exiting log stream.
2026-03-15 04:53:41.422 DEBUG [src.node.waku_node] Container stopped.
2026-03-15 04:53:41.424 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-15 04:53:41.439 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-15 04:53:41.445 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-15 04:53:41.453 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-15 04:53:41.457 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-15 04:53:41.461 DEBUG [src.node.docker_mananger] No errors found in the waku logs.