198 lines
36 KiB
Plaintext

2026-01-02 04:27:22.669 DEBUG [tests.conftest] Running fixture setup: test_id
2026-01-02 04:27:22.669 DEBUG [tests.conftest] Running test: test_relay_get_message_while_one_peer_is_paused with id: 2026-01-02_04-27-22__9b54d5f5-7023-414d-a8ba-71eb63fff646
2026-01-02 04:27:22.670 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-01-02 04:27:22.670 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-01-02 04:27:22.670 DEBUG [src.steps.relay] Running fixture setup: setup_main_relay_nodes
2026-01-02 04:27:22.677 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-02 04:27:22.677 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-01-02_04-27-22__9b54d5f5-7023-414d-a8ba-71eb63fff646__wakuorg_nwaku:latest.log
2026-01-02 04:27:22.677 DEBUG [src.node.waku_node] Starting Node...
2026-01-02 04:27:22.677 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-02 04:27:22.679 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-02 04:27:22.679 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.207.231
2026-01-02 04:27:22.679 DEBUG [src.node.docker_mananger] Generated ports ['3368', '3369', '3370', '3371', '3372']
2026-01-02 04:27:22.679 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-02 04:27:22.679 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-02 04:27:22.679 DEBUG [src.node.waku_node] Using volumes []
2026-01-02 04:27:22.680 DEBUG [src.node.docker_mananger] docker run -i -t -p 3368:3368 -p 3369:3369 -p 3370:3370 -p 3371:3371 -p 3372:3372 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=3370 --rest-port=3368 --tcp-port=3369 --discv5-udp-port=3371 --rest-address=0.0.0.0 --nat=extip:172.18.207.231 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=80971dd9a9afa36966d456eb87f21ba4f0a2b331f46ee00e78fdd34cd97dadc8 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=3372 --metrics-logging=true --relay=true
2026-01-02 04:27:22.868 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.207.231 waku 58d163354f1f198689e9faa8b90aedb4a3071a09402ae365ed768a8359c0a2fd
2026-01-02 04:27:22.907 DEBUG [src.node.docker_mananger] Container started with ID 58d163354f1f. Setting up logs at ./log/docker/node1_2026-01-02_04-27-22__9b54d5f5-7023-414d-a8ba-71eb63fff646__wakuorg_nwaku:latest.log
2026-01-02 04:27:22.907 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 3368
2026-01-02 04:27:22.908 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-02 04:27:23.909 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:3368/health" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:23.912 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-02 04:27:23.913 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-02 04:27:23.913 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:3368/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:23.915 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.207.231/tcp/3369/p2p/16Uiu2HAkxJ6ybmfh3wkFx4FiUkhzSLBD7g59LVyvVWoEdb34phh3","/ip4/172.18.207.231/tcp/3370/ws/p2p/16Uiu2HAkxJ6ybmfh3wkFx4FiUkhzSLBD7g59LVyvVWoEdb34phh3"],"enrUri":"enr:-L24QAJH4VatEtKwV7pETNurz08ymRRK8keE28L4AgEy8wJ0NcFq_OW1J4EsSOZoe89c7IvdN83PjjBCW8OBikIsQZICgmlkgnY0gmlwhKwSz-eKbXVsdGlhZGRyc5YACASsEs_nBg0pAAoErBLP5wYNKt0DgnJzhQADAQAAiXNlY3AyNTZrMaECKqKtmesPCKyvkwTSl7ypI3CN-yBrgxkl3CCVxjJFYfqDdGNwgg0pg3VkcIINK4V3YWt1MgE"}'
2026-01-02 04:27:23.915 INFO [src.node.waku_node] REST service is ready !!
2026-01-02 04:27:23.922 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-02 04:27:23.922 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-01-02_04-27-22__9b54d5f5-7023-414d-a8ba-71eb63fff646__wakuorg_nwaku:latest.log
2026-01-02 04:27:23.922 DEBUG [src.node.waku_node] Starting Node...
2026-01-02 04:27:23.922 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-02 04:27:23.924 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-02 04:27:23.924 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.55.63
2026-01-02 04:27:23.924 DEBUG [src.node.docker_mananger] Generated ports ['33304', '33305', '33306', '33307', '33308']
2026-01-02 04:27:23.924 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-02 04:27:23.925 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-02 04:27:23.925 DEBUG [src.node.waku_node] Using volumes []
2026-01-02 04:27:23.925 DEBUG [src.node.docker_mananger] docker run -i -t -p 33304:33304 -p 33305:33305 -p 33306:33306 -p 33307:33307 -p 33308:33308 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=33306 --rest-port=33304 --tcp-port=33305 --discv5-udp-port=33307 --rest-address=0.0.0.0 --nat=extip:172.18.55.63 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=caaf92a7f27cce3a8effdeaed508dae5bda500e6d4bbb8adfe462caa3fef9b1f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=33308 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QAJH4VatEtKwV7pETNurz08ymRRK8keE28L4AgEy8wJ0NcFq_OW1J4EsSOZoe89c7IvdN83PjjBCW8OBikIsQZICgmlkgnY0gmlwhKwSz-eKbXVsdGlhZGRyc5YACASsEs_nBg0pAAoErBLP5wYNKt0DgnJzhQADAQAAiXNlY3AyNTZrMaECKqKtmesPCKyvkwTSl7ypI3CN-yBrgxkl3CCVxjJFYfqDdGNwgg0pg3VkcIINK4V3YWt1MgE
2026-01-02 04:27:24.108 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.55.63 waku cb83cf7be23529027e3a7bbc7ed5b1360efbe904202daa80caca5e362a29af28
2026-01-02 04:27:24.139 DEBUG [src.node.docker_mananger] Container started with ID cb83cf7be235. Setting up logs at ./log/docker/node2_2026-01-02_04-27-22__9b54d5f5-7023-414d-a8ba-71eb63fff646__wakuorg_nwaku:latest.log
2026-01-02 04:27:24.139 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 33304
2026-01-02 04:27:24.139 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-02 04:27:25.141 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33304/health" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:25.150 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-02 04:27:25.152 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-02 04:27:25.154 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33304/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:25.160 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.55.63/tcp/33305/p2p/16Uiu2HAmRzBsXjyP8W1G9LmqSiaeMeRZtvRxDtL5xyvyChrX5ewH","/ip4/172.18.55.63/tcp/33306/ws/p2p/16Uiu2HAmRzBsXjyP8W1G9LmqSiaeMeRZtvRxDtL5xyvyChrX5ewH"],"enrUri":"enr:-L24QEQUi3tBgLKWFbPY39Eqtfh1GiW3ngVORoYqGgmlU3v1NPWvcX3RmWCsLmmAtLkgEu24nU75lZLzt00_kwol1dUCgmlkgnY0gmlwhKwSNz-KbXVsdGlhZGRyc5YACASsEjc_BoIZAAoErBI3PwaCGt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDxhL-IYF-zH65gQgpje1tRvM0BZm_8cPfgNXmRfJVhiCDdGNwgoIZg3VkcIKCG4V3YWt1MgE"}'
2026-01-02 04:27:25.160 INFO [src.node.waku_node] REST service is ready !!
2026-01-02 04:27:25.161 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:33304/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.207.231/tcp/3369/p2p/16Uiu2HAkxJ6ybmfh3wkFx4FiUkhzSLBD7g59LVyvVWoEdb34phh3"]'
2026-01-02 04:27:25.163 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:25.163 DEBUG [src.steps.relay] Running fixture setup: setup_optional_relay_nodes
2026-01-02 04:27:25.170 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-02 04:27:25.170 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-01-02_04-27-22__9b54d5f5-7023-414d-a8ba-71eb63fff646__wakuorg_nwaku:latest.log
2026-01-02 04:27:25.170 DEBUG [src.node.waku_node] Starting Node...
2026-01-02 04:27:25.170 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-02 04:27:25.171 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-02 04:27:25.171 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.215.116
2026-01-02 04:27:25.171 DEBUG [src.node.docker_mananger] Generated ports ['46883', '46884', '46885', '46886', '46887']
2026-01-02 04:27:25.172 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-02 04:27:25.172 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-02 04:27:25.172 DEBUG [src.node.waku_node] Using volumes []
2026-01-02 04:27:25.172 DEBUG [src.node.docker_mananger] docker run -i -t -p 46883:46883 -p 46884:46884 -p 46885:46885 -p 46886:46886 -p 46887:46887 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=46885 --rest-port=46883 --tcp-port=46884 --discv5-udp-port=46886 --rest-address=0.0.0.0 --nat=extip:172.18.215.116 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=34caf2961f7e2b3aada8ceee3cd42bfecdf9ab55519b811d7e3e56ee33d22808 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=46887 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QAJH4VatEtKwV7pETNurz08ymRRK8keE28L4AgEy8wJ0NcFq_OW1J4EsSOZoe89c7IvdN83PjjBCW8OBikIsQZICgmlkgnY0gmlwhKwSz-eKbXVsdGlhZGRyc5YACASsEs_nBg0pAAoErBLP5wYNKt0DgnJzhQADAQAAiXNlY3AyNTZrMaECKqKtmesPCKyvkwTSl7ypI3CN-yBrgxkl3CCVxjJFYfqDdGNwgg0pg3VkcIINK4V3YWt1MgE
2026-01-02 04:27:25.348 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.215.116 waku fe8da6e4acb98904146329de4d13189bbc02e3611447c91021abeed405be5002
2026-01-02 04:27:25.378 DEBUG [src.node.docker_mananger] Container started with ID fe8da6e4acb9. Setting up logs at ./log/docker/node3_2026-01-02_04-27-22__9b54d5f5-7023-414d-a8ba-71eb63fff646__wakuorg_nwaku:latest.log
2026-01-02 04:27:25.379 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 46883
2026-01-02 04:27:25.379 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-02 04:27:26.381 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:46883/health" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:26.391 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-02 04:27:26.392 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-02 04:27:26.392 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:46883/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:26.396 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.215.116/tcp/46884/p2p/16Uiu2HAkus6jUGUU18ZuY7fFJvZUhtQrcaTTSnsouNMmkggE6SN3","/ip4/172.18.215.116/tcp/46885/ws/p2p/16Uiu2HAkus6jUGUU18ZuY7fFJvZUhtQrcaTTSnsouNMmkggE6SN3"],"enrUri":"enr:-L24QCbmvsUVkyrvu_r1IzFssycccE--IsMEpfgQcv31MaN6GxngSZv5afNSbyckCuxzFbPCONzUO2zZls9fSZ0gpGsCgmlkgnY0gmlwhKwS13SKbXVsdGlhZGRyc5YACASsEtd0BrckAAoErBLXdAa3Jd0DgnJzhQADAQAAiXNlY3AyNTZrMaECBoOCM5jMdP_tSpDT-oBiw6G3N6K_Zz2KVSMuWMYoRICDdGNwgrckg3VkcIK3JoV3YWt1MgE"}'
2026-01-02 04:27:26.396 INFO [src.node.waku_node] REST service is ready !!
2026-01-02 04:27:26.397 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:46883/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.207.231/tcp/3369/p2p/16Uiu2HAkxJ6ybmfh3wkFx4FiUkhzSLBD7g59LVyvVWoEdb34phh3"]'
2026-01-02 04:27:26.401 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:26.408 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-02 04:27:26.408 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-01-02_04-27-22__9b54d5f5-7023-414d-a8ba-71eb63fff646__wakuorg_nwaku:latest.log
2026-01-02 04:27:26.408 DEBUG [src.node.waku_node] Starting Node...
2026-01-02 04:27:26.408 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-02 04:27:26.409 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-02 04:27:26.410 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.59.97
2026-01-02 04:27:26.410 DEBUG [src.node.docker_mananger] Generated ports ['36867', '36868', '36869', '36870', '36871']
2026-01-02 04:27:26.410 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-02 04:27:26.410 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-02 04:27:26.410 DEBUG [src.node.waku_node] Using volumes []
2026-01-02 04:27:26.410 DEBUG [src.node.docker_mananger] docker run -i -t -p 36867:36867 -p 36868:36868 -p 36869:36869 -p 36870:36870 -p 36871:36871 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=36869 --rest-port=36867 --tcp-port=36868 --discv5-udp-port=36870 --rest-address=0.0.0.0 --nat=extip:172.18.59.97 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=e7b8dd0dccdc9bc5e8b83e20236ee6ebcabb5aa9d79b703fca5bb78f03eaab9e --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=36871 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QAJH4VatEtKwV7pETNurz08ymRRK8keE28L4AgEy8wJ0NcFq_OW1J4EsSOZoe89c7IvdN83PjjBCW8OBikIsQZICgmlkgnY0gmlwhKwSz-eKbXVsdGlhZGRyc5YACASsEs_nBg0pAAoErBLP5wYNKt0DgnJzhQADAQAAiXNlY3AyNTZrMaECKqKtmesPCKyvkwTSl7ypI3CN-yBrgxkl3CCVxjJFYfqDdGNwgg0pg3VkcIINK4V3YWt1MgE
2026-01-02 04:27:26.592 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.59.97 waku 3d53675ccc4da0ed4aff643750565e39f6fe754aa3d465853c281c145c2dd4f0
2026-01-02 04:27:26.622 DEBUG [src.node.docker_mananger] Container started with ID 3d53675ccc4d. Setting up logs at ./log/docker/node4_2026-01-02_04-27-22__9b54d5f5-7023-414d-a8ba-71eb63fff646__wakuorg_nwaku:latest.log
2026-01-02 04:27:26.622 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 36867
2026-01-02 04:27:26.623 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-02 04:27:27.623 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36867/health" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:27.632 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-02 04:27:27.633 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-02 04:27:27.634 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36867/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:27.639 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.59.97/tcp/36868/p2p/16Uiu2HAmF1UxpsMA5w62hzDQBUn9pnLZew7XAvdPtbPrW1LQdqSU","/ip4/172.18.59.97/tcp/36869/ws/p2p/16Uiu2HAmF1UxpsMA5w62hzDQBUn9pnLZew7XAvdPtbPrW1LQdqSU"],"enrUri":"enr:-L24QOqNF7yzhh5kXX9YHkFNziz0K9AbkhecwYT_YIcI2YNUb5bieGmXzq3K5JGbh5EP5Lx5QNvMDvQfTATO-yo0Kp8CgmlkgnY0gmlwhKwSO2GKbXVsdGlhZGRyc5YACASsEjthBpAEAAoErBI7YQaQBd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDIvdXrsPEXa9c_52EHQOmmVUHqdW7WSgsbrF-V3tHLvWDdGNwgpAEg3VkcIKQBoV3YWt1MgE"}'
2026-01-02 04:27:27.639 INFO [src.node.waku_node] REST service is ready !!
2026-01-02 04:27:27.640 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:36867/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.207.231/tcp/3369/p2p/16Uiu2HAkxJ6ybmfh3wkFx4FiUkhzSLBD7g59LVyvVWoEdb34phh3"]'
2026-01-02 04:27:27.642 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:27.649 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-02 04:27:27.649 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-01-02_04-27-22__9b54d5f5-7023-414d-a8ba-71eb63fff646__wakuorg_nwaku:latest.log
2026-01-02 04:27:27.649 DEBUG [src.node.waku_node] Starting Node...
2026-01-02 04:27:27.649 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-02 04:27:27.650 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-02 04:27:27.650 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.181.38
2026-01-02 04:27:27.650 DEBUG [src.node.docker_mananger] Generated ports ['8434', '8435', '8436', '8437', '8438']
2026-01-02 04:27:27.651 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-02 04:27:27.651 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-02 04:27:27.651 DEBUG [src.node.waku_node] Using volumes []
2026-01-02 04:27:27.651 DEBUG [src.node.docker_mananger] docker run -i -t -p 8434:8434 -p 8435:8435 -p 8436:8436 -p 8437:8437 -p 8438:8438 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=8436 --rest-port=8434 --tcp-port=8435 --discv5-udp-port=8437 --rest-address=0.0.0.0 --nat=extip:172.18.181.38 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=4442bc7bd5e30104b1c64beab824acd1d546dfbb8bcad691451c2cb2f51ab3f2 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=8438 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QAJH4VatEtKwV7pETNurz08ymRRK8keE28L4AgEy8wJ0NcFq_OW1J4EsSOZoe89c7IvdN83PjjBCW8OBikIsQZICgmlkgnY0gmlwhKwSz-eKbXVsdGlhZGRyc5YACASsEs_nBg0pAAoErBLP5wYNKt0DgnJzhQADAQAAiXNlY3AyNTZrMaECKqKtmesPCKyvkwTSl7ypI3CN-yBrgxkl3CCVxjJFYfqDdGNwgg0pg3VkcIINK4V3YWt1MgE
2026-01-02 04:27:27.836 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.181.38 waku c1ab8e913956e219ccaf84d3706383e666927b9e53e72111d7b3225f9fe98854
2026-01-02 04:27:27.866 DEBUG [src.node.docker_mananger] Container started with ID c1ab8e913956. Setting up logs at ./log/docker/node5_2026-01-02_04-27-22__9b54d5f5-7023-414d-a8ba-71eb63fff646__wakuorg_nwaku:latest.log
2026-01-02 04:27:27.866 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 8434
2026-01-02 04:27:27.867 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-02 04:27:28.867 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8434/health" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:28.875 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-02 04:27:28.876 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-02 04:27:28.877 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8434/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:28.880 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.181.38/tcp/8435/p2p/16Uiu2HAm77BfQsVqW13n19BiqLg5a3z2vceisVLR845WWNuGJe93","/ip4/172.18.181.38/tcp/8436/ws/p2p/16Uiu2HAm77BfQsVqW13n19BiqLg5a3z2vceisVLR845WWNuGJe93"],"enrUri":"enr:-L24QJ_0oVWj0vRj5YEeYuIcuG3rDLKgbx557-XrXMTZFdYZBOOqjwNQ6I5r0HqixnD7l2NrL9_UPEWwskgWBi4-qr4CgmlkgnY0gmlwhKwStSaKbXVsdGlhZGRyc5YACASsErUmBiDzAAoErBK1JgYg9N0DgnJzhQADAQAAiXNlY3AyNTZrMaECrY_FQGKYw4LibENeD3dsw1-z6Z0670fc3aslcYTkid6DdGNwgiDzg3VkcIIg9YV3YWt1MgE"}'
2026-01-02 04:27:28.881 INFO [src.node.waku_node] REST service is ready !!
2026-01-02 04:27:28.882 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:8434/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.207.231/tcp/3369/p2p/16Uiu2HAkxJ6ybmfh3wkFx4FiUkhzSLBD7g59LVyvVWoEdb34phh3"]'
2026-01-02 04:27:28.886 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:28.886 DEBUG [src.steps.relay] Running fixture setup: subscribe_main_relay_nodes
2026-01-02 04:27:28.887 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:3368/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-02 04:27:28.892 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:28.893 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:33304/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-02 04:27:28.899 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:28.900 DEBUG [src.steps.relay] Running fixture setup: subscribe_optional_relay_nodes
2026-01-02 04:27:28.900 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:46883/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-02 04:27:28.905 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:28.907 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:36867/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-02 04:27:28.912 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:28.912 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:8434/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-02 04:27:28.917 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:28.918 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:3368/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-01-02 04:27:28.930 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:28.930 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-01-02 04:27:29.030 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-01-02 04:27:29.030 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:3368/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:29.032 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767328048918678016,"ephemeral":false,"proof":""}]'
2026-01-02 04:27:29.034 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-01-02 04:27:29.034 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33304/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:29.036 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767328048918678016,"ephemeral":false,"proof":""}]'
2026-01-02 04:27:29.038 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-01-02 04:27:29.038 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:46883/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:29.040 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767328048918678016,"ephemeral":false,"proof":""}]'
2026-01-02 04:27:29.041 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-01-02 04:27:29.041 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36867/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:29.043 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767328048918678016,"ephemeral":false,"proof":""}]'
2026-01-02 04:27:29.045 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-01-02 04:27:29.045 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8434/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:29.047 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767328048918678016,"ephemeral":false,"proof":""}]'
2026-01-02 04:27:29.048 INFO [src.steps.relay] WARM UP successful!!
2026-01-02 04:27:29.050 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:3368/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-01-02 04:27:29.056 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:29.057 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-01-02 04:27:29.157 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-01-02 04:27:29.157 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:3368/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:29.160 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767328049050559755,"ephemeral":false,"proof":""}]'
2026-01-02 04:27:29.161 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-01-02 04:27:29.161 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33304/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:29.163 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767328049050559755,"ephemeral":false,"proof":""}]'
2026-01-02 04:27:29.164 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-01-02 04:27:29.165 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:46883/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:29.167 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767328049050559755,"ephemeral":false,"proof":""}]'
2026-01-02 04:27:29.168 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-01-02 04:27:29.168 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36867/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:29.170 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767328049050559755,"ephemeral":false,"proof":""}]'
2026-01-02 04:27:29.172 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-01-02 04:27:29.172 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:8434/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:29.174 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767328049050559755,"ephemeral":false,"proof":""}]'
2026-01-02 04:27:29.175 DEBUG [src.node.waku_node] Pausing container with id cb83cf7be235
2026-01-02 04:27:29.184 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:3368/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-01-02 04:27:29.191 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:29.191 DEBUG [src.node.waku_node] Unpause container with id cb83cf7be235
2026-01-02 04:27:29.201 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:3368/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-01-02 04:27:29.207 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:27:29.210 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33304/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:27:29.213 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767328049175365362,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767328049175436694,"ephemeral":false,"proof":""}]'
2026-01-02 04:27:29.216 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-01-02 04:27:29.217 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-01-02 04:27:29.217 DEBUG [src.node.waku_node] Stopping container with id 58d163354f1f
2026-01-02 04:27:29.778 DEBUG [src.node.waku_node] Container stopped.
2026-01-02 04:27:29.780 DEBUG [src.node.waku_node] Stopping container with id cb83cf7be235
2026-01-02 04:27:30.357 DEBUG [src.node.waku_node] Container stopped.
2026-01-02 04:27:30.357 DEBUG [src.node.waku_node] Stopping container with id fe8da6e4acb9
2026-01-02 04:27:30.591 ERROR [src.node.docker_mananger] Max retries reached for container 58d163354f1f. Exiting log stream.
2026-01-02 04:27:30.846 DEBUG [src.node.waku_node] Container stopped.
2026-01-02 04:27:30.848 DEBUG [src.node.waku_node] Stopping container with id 3d53675ccc4d
2026-01-02 04:27:31.171 ERROR [src.node.docker_mananger] Max retries reached for container cb83cf7be235. Exiting log stream.
2026-01-02 04:27:31.383 DEBUG [src.node.waku_node] Container stopped.
2026-01-02 04:27:31.383 DEBUG [src.node.waku_node] Stopping container with id c1ab8e913956
2026-01-02 04:27:31.659 ERROR [src.node.docker_mananger] Max retries reached for container fe8da6e4acb9. Exiting log stream.
2026-01-02 04:27:31.876 DEBUG [src.node.waku_node] Container stopped.
2026-01-02 04:27:31.877 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-01-02 04:27:31.893 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-02 04:27:31.899 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-02 04:27:31.906 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-02 04:27:31.911 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-02 04:27:31.917 DEBUG [src.node.docker_mananger] No errors found in the waku logs.