168 lines
32 KiB
Plaintext

2026-03-03 04:33:51.506 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-03 04:33:51.506 DEBUG [tests.conftest] Running test: test_all_nodes_subscribed_to_the_topic with id: 2026-03-03_04-33-51__f07d47f2-84fe-418f-b536-378d7a581a5d
2026-03-03 04:33:51.507 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-03 04:33:51.507 DEBUG [src.steps.filter] Running fixture setup: filter_setup
2026-03-03 04:33:51.507 DEBUG [src.steps.filter] Running fixture setup: setup_main_relay_node
2026-03-03 04:33:51.514 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:33:51.514 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-03_04-33-51__f07d47f2-84fe-418f-b536-378d7a581a5d__wakuorg_nwaku:latest.log
2026-03-03 04:33:51.514 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:33:51.514 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:33:51.516 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:33:51.516 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.123.131
2026-03-03 04:33:51.516 DEBUG [src.node.docker_mananger] Generated ports ['63585', '63586', '63587', '63588', '63589']
2026-03-03 04:33:51.516 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:33:51.516 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:33:51.516 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:33:51.516 DEBUG [src.node.docker_mananger] docker run -i -t -p 63585:63585 -p 63586:63586 -p 63587:63587 -p 63588:63588 -p 63589:63589 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=63587 --rest-port=63585 --tcp-port=63586 --discv5-udp-port=63588 --rest-address=0.0.0.0 --nat=extip:172.18.123.131 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=eda9a8ab28de56bca9746dab5be49e22dcd199cd9ecbc58bd4ae9ff477ba77f8 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=63589 --metrics-logging=true --relay=true --filter=true
2026-03-03 04:33:51.716 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.123.131 waku c76fc5ef62944e70e785e5674b5105a5713b352d76b4de5e9d0b6805b43547eb
2026-03-03 04:33:51.749 ERROR [src.node.docker_mananger] Max retries reached for container bb5431a5ef73. Exiting log stream.
2026-03-03 04:33:51.751 DEBUG [src.node.docker_mananger] Container started with ID c76fc5ef6294. Setting up logs at ./log/docker/node1_2026-03-03_04-33-51__f07d47f2-84fe-418f-b536-378d7a581a5d__wakuorg_nwaku:latest.log
2026-03-03 04:33:51.752 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 63585
2026-03-03 04:33:51.753 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:33:52.177 ERROR [src.node.docker_mananger] Max retries reached for container dc239d066777. Exiting log stream.
2026-03-03 04:33:52.754 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63585/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:52.757 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:33:52.757 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:33:52.757 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63585/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:52.759 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.123.131/tcp/63586/p2p/16Uiu2HAmFRj6JW5PnzAfxJ26amykxwwsoYj7uXihMpvkYumyv8Fh","/ip4/172.18.123.131/tcp/63587/ws/p2p/16Uiu2HAmFRj6JW5PnzAfxJ26amykxwwsoYj7uXihMpvkYumyv8Fh"],"enrUri":"enr:-L24QO-lPhQSXnp5h2yQGdwCxsoKJCY6gZyQVIgrwjwSxj8SWPD0o4RYLTKsPpkDzMToUwAUfll--FR1lPjehY4ATw0CgmlkgnY0gmlwhKwSe4OKbXVsdGlhZGRyc5YACASsEnuDBvhiAAoErBJ7gwb4Y90DgnJzhQADAQAAiXNlY3AyNTZrMaEDKS1B7Y0m1CSHbyolLaLXaqhxbF5Nw8VtEm_wtGpC0BiDdGNwgvhig3VkcIL4ZIV3YWt1MgU"}'
2026-03-03 04:33:52.760 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:33:52.760 DEBUG [src.steps.filter] Running fixture setup: setup_main_filter_node
2026-03-03 04:33:52.767 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:33:52.767 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-03_04-33-51__f07d47f2-84fe-418f-b536-378d7a581a5d__wakuorg_nwaku:latest.log
2026-03-03 04:33:52.767 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:33:52.767 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:33:52.768 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:33:52.768 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.186.17
2026-03-03 04:33:52.769 DEBUG [src.node.docker_mananger] Generated ports ['50711', '50712', '50713', '50714', '50715']
2026-03-03 04:33:52.769 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:33:52.769 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:33:52.769 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:33:52.769 DEBUG [src.node.docker_mananger] docker run -i -t -p 50711:50711 -p 50712:50712 -p 50713:50713 -p 50714:50714 -p 50715:50715 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=50713 --rest-port=50711 --tcp-port=50712 --discv5-udp-port=50714 --rest-address=0.0.0.0 --nat=extip:172.18.186.17 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=c8fe4d8fe1ef5c3cf0fbaa9dbaaec79ba69fcbfe73e7f7e396fab6c284fde6cf --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=50715 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QO-lPhQSXnp5h2yQGdwCxsoKJCY6gZyQVIgrwjwSxj8SWPD0o4RYLTKsPpkDzMToUwAUfll--FR1lPjehY4ATw0CgmlkgnY0gmlwhKwSe4OKbXVsdGlhZGRyc5YACASsEnuDBvhiAAoErBJ7gwb4Y90DgnJzhQADAQAAiXNlY3AyNTZrMaEDKS1B7Y0m1CSHbyolLaLXaqhxbF5Nw8VtEm_wtGpC0BiDdGNwgvhig3VkcIL4ZIV3YWt1MgU --filternode=/ip4/172.18.123.131/tcp/63586/p2p/16Uiu2HAmFRj6JW5PnzAfxJ26amykxwwsoYj7uXihMpvkYumyv8Fh
2026-03-03 04:33:52.970 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.186.17 waku bd0ebd71ad09e2e1e3961bf28db3239f25231865776b57f24dddc1801d1315f7
2026-03-03 04:33:53.007 DEBUG [src.node.docker_mananger] Container started with ID bd0ebd71ad09. Setting up logs at ./log/docker/node2_2026-03-03_04-33-51__f07d47f2-84fe-418f-b536-378d7a581a5d__wakuorg_nwaku:latest.log
2026-03-03 04:33:53.007 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 50711
2026-03-03 04:33:53.009 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:33:54.009 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50711/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:54.012 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:33:54.012 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:33:54.013 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50711/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:54.015 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.186.17/tcp/50712/p2p/16Uiu2HAm1EzrQhfE6kX2y6FoDVQggr9BKMoHEhJfoWkZP5R8jF8d","/ip4/172.18.186.17/tcp/50713/ws/p2p/16Uiu2HAm1EzrQhfE6kX2y6FoDVQggr9BKMoHEhJfoWkZP5R8jF8d"],"enrUri":"enr:-L24QOlmg26uglYX1sOzUU6GoKgdMTtWZ9XBUvc1GJtmcX4mRXQtTDvjn9qevoIDuOqCSlic3hR6Va5PaDiGNYbu9lICgmlkgnY0gmlwhKwSuhGKbXVsdGlhZGRyc5YACASsEroRBsYYAAoErBK6EQbGGd0DgnJzhQADAQAAiXNlY3AyNTZrMaECVmoSc4bTeahTOcVBjmK2QMrJtGzbgeoTeiWV__VcqLKDdGNwgsYYg3VkcILGGoV3YWt1MgA"}'
2026-03-03 04:33:54.015 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:33:54.015 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:50711/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.123.131/tcp/63586/p2p/16Uiu2HAmFRj6JW5PnzAfxJ26amykxwwsoYj7uXihMpvkYumyv8Fh"]'
2026-03-03 04:33:54.053 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:33:54.065 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:33:54.065 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-03_04-33-51__f07d47f2-84fe-418f-b536-378d7a581a5d__wakuorg_nwaku:latest.log
2026-03-03 04:33:54.066 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:33:54.066 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:33:54.067 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:33:54.067 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.212.239
2026-03-03 04:33:54.068 DEBUG [src.node.docker_mananger] Generated ports ['43805', '43806', '43807', '43808', '43809']
2026-03-03 04:33:54.068 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:33:54.068 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:33:54.068 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:33:54.068 DEBUG [src.node.docker_mananger] docker run -i -t -p 43805:43805 -p 43806:43806 -p 43807:43807 -p 43808:43808 -p 43809:43809 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=43807 --rest-port=43805 --tcp-port=43806 --discv5-udp-port=43808 --rest-address=0.0.0.0 --nat=extip:172.18.212.239 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=fec110d2c4dfe730b5a81596d409aeef9f82a97c7d495dd74edaf06c7bfed1c9 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=43809 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QO-lPhQSXnp5h2yQGdwCxsoKJCY6gZyQVIgrwjwSxj8SWPD0o4RYLTKsPpkDzMToUwAUfll--FR1lPjehY4ATw0CgmlkgnY0gmlwhKwSe4OKbXVsdGlhZGRyc5YACASsEnuDBvhiAAoErBJ7gwb4Y90DgnJzhQADAQAAiXNlY3AyNTZrMaEDKS1B7Y0m1CSHbyolLaLXaqhxbF5Nw8VtEm_wtGpC0BiDdGNwgvhig3VkcIL4ZIV3YWt1MgU --filternode=/ip4/172.18.123.131/tcp/63586/p2p/16Uiu2HAmFRj6JW5PnzAfxJ26amykxwwsoYj7uXihMpvkYumyv8Fh
2026-03-03 04:33:54.280 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.212.239 waku 3300d18e72fd5ca1350842ca76070ab688a5e9cffeb7fc471e8133c496afde25
2026-03-03 04:33:54.314 DEBUG [src.node.docker_mananger] Container started with ID 3300d18e72fd. Setting up logs at ./log/docker/node3_2026-03-03_04-33-51__f07d47f2-84fe-418f-b536-378d7a581a5d__wakuorg_nwaku:latest.log
2026-03-03 04:33:54.314 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 43805
2026-03-03 04:33:54.314 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:33:55.315 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43805/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:55.318 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:33:55.318 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:33:55.318 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43805/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:55.320 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.212.239/tcp/43806/p2p/16Uiu2HAkv1qHf9Zdot4tazAE3DR1PGhK9Tb2gMDRZ7KbRwxNt4zo","/ip4/172.18.212.239/tcp/43807/ws/p2p/16Uiu2HAkv1qHf9Zdot4tazAE3DR1PGhK9Tb2gMDRZ7KbRwxNt4zo"],"enrUri":"enr:-L24QBpGssWxPPUgOl9wcivJiCD44jc5JlgSAsuTgioRg-nZP17pkX-LlMLbZIOyPWRo2xra0drya2SHa0xxZFP32S0CgmlkgnY0gmlwhKwS1O-KbXVsdGlhZGRyc5YACASsEtTvBqseAAoErBLU7warH90DgnJzhQADAQAAiXNlY3AyNTZrMaECCMBFqX6k2CVX5syoBY81yg5f29tuyj50wi4lc9EgHayDdGNwgqseg3VkcIKrIIV3YWt1MgA"}'
2026-03-03 04:33:55.321 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:33:55.321 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:43805/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.123.131/tcp/63586/p2p/16Uiu2HAmFRj6JW5PnzAfxJ26amykxwwsoYj7uXihMpvkYumyv8Fh"]'
2026-03-03 04:33:55.355 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:33:55.368 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:33:55.369 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-03_04-33-51__f07d47f2-84fe-418f-b536-378d7a581a5d__wakuorg_nwaku:latest.log
2026-03-03 04:33:55.369 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:33:55.369 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:33:55.373 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:33:55.375 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.83.96
2026-03-03 04:33:55.376 DEBUG [src.node.docker_mananger] Generated ports ['7138', '7139', '7140', '7141', '7142']
2026-03-03 04:33:55.376 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:33:55.376 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:33:55.377 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:33:55.377 DEBUG [src.node.docker_mananger] docker run -i -t -p 7138:7138 -p 7139:7139 -p 7140:7140 -p 7141:7141 -p 7142:7142 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=7140 --rest-port=7138 --tcp-port=7139 --discv5-udp-port=7141 --rest-address=0.0.0.0 --nat=extip:172.18.83.96 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=d5bac38ff0b3498c686fb232a0201eaa530aa781fa0fefe13ef09fc1ae6fb0eb --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=7142 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QO-lPhQSXnp5h2yQGdwCxsoKJCY6gZyQVIgrwjwSxj8SWPD0o4RYLTKsPpkDzMToUwAUfll--FR1lPjehY4ATw0CgmlkgnY0gmlwhKwSe4OKbXVsdGlhZGRyc5YACASsEnuDBvhiAAoErBJ7gwb4Y90DgnJzhQADAQAAiXNlY3AyNTZrMaEDKS1B7Y0m1CSHbyolLaLXaqhxbF5Nw8VtEm_wtGpC0BiDdGNwgvhig3VkcIL4ZIV3YWt1MgU --filternode=/ip4/172.18.123.131/tcp/63586/p2p/16Uiu2HAmFRj6JW5PnzAfxJ26amykxwwsoYj7uXihMpvkYumyv8Fh
2026-03-03 04:33:55.573 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.83.96 waku e3ad47f76b7d416d61592d5a460acac297129e547565418031234d2166df3e4f
2026-03-03 04:33:55.606 DEBUG [src.node.docker_mananger] Container started with ID e3ad47f76b7d. Setting up logs at ./log/docker/node4_2026-03-03_04-33-51__f07d47f2-84fe-418f-b536-378d7a581a5d__wakuorg_nwaku:latest.log
2026-03-03 04:33:55.606 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 7138
2026-03-03 04:33:55.606 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:33:56.607 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7138/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:56.610 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:33:56.610 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:33:56.610 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7138/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:56.613 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.83.96/tcp/7139/p2p/16Uiu2HAkyMpAbtpiVmrVA93eao6pzUDNjXewdLYSLTUAsVSHoDqU","/ip4/172.18.83.96/tcp/7140/ws/p2p/16Uiu2HAkyMpAbtpiVmrVA93eao6pzUDNjXewdLYSLTUAsVSHoDqU"],"enrUri":"enr:-L24QPf3q7APLzVP7VjmjdHrUQIYU9g1x4c9-T9YWfa2WgsqRHG8td7t8WBPIhgYrotmKPefdXIEHivL6sZtVANDrd0CgmlkgnY0gmlwhKwSU2CKbXVsdGlhZGRyc5YACASsElNgBhvjAAoErBJTYAYb5N0DgnJzhQADAQAAiXNlY3AyNTZrMaECOnGuSf0tZnJVgxIuHykh88IiJDTHbPXdUB2H4zWk_HuDdGNwghvjg3VkcIIb5YV3YWt1MgA"}'
2026-03-03 04:33:56.613 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:33:56.613 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7138/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.123.131/tcp/63586/p2p/16Uiu2HAmFRj6JW5PnzAfxJ26amykxwwsoYj7uXihMpvkYumyv8Fh"]'
2026-03-03 04:33:56.650 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:33:56.663 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:33:56.664 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-03-03_04-33-51__f07d47f2-84fe-418f-b536-378d7a581a5d__wakuorg_nwaku:latest.log
2026-03-03 04:33:56.664 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:33:56.664 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:33:56.667 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:33:56.667 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.178.169
2026-03-03 04:33:56.668 DEBUG [src.node.docker_mananger] Generated ports ['33111', '33112', '33113', '33114', '33115']
2026-03-03 04:33:56.668 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:33:56.668 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:33:56.669 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:33:56.669 DEBUG [src.node.docker_mananger] docker run -i -t -p 33111:33111 -p 33112:33112 -p 33113:33113 -p 33114:33114 -p 33115:33115 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=33113 --rest-port=33111 --tcp-port=33112 --discv5-udp-port=33114 --rest-address=0.0.0.0 --nat=extip:172.18.178.169 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=f4b3bdcfe57628ff53eed0cf613e944af52e88a59fc6f8aef586c0aa89b778af --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=33115 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QO-lPhQSXnp5h2yQGdwCxsoKJCY6gZyQVIgrwjwSxj8SWPD0o4RYLTKsPpkDzMToUwAUfll--FR1lPjehY4ATw0CgmlkgnY0gmlwhKwSe4OKbXVsdGlhZGRyc5YACASsEnuDBvhiAAoErBJ7gwb4Y90DgnJzhQADAQAAiXNlY3AyNTZrMaEDKS1B7Y0m1CSHbyolLaLXaqhxbF5Nw8VtEm_wtGpC0BiDdGNwgvhig3VkcIL4ZIV3YWt1MgU --filternode=/ip4/172.18.123.131/tcp/63586/p2p/16Uiu2HAmFRj6JW5PnzAfxJ26amykxwwsoYj7uXihMpvkYumyv8Fh
2026-03-03 04:33:56.870 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.178.169 waku a7076fabbec1593ec8e627e21ebe5fff657b0beb615558db1b88f1e8068dd65d
2026-03-03 04:33:56.903 DEBUG [src.node.docker_mananger] Container started with ID a7076fabbec1. Setting up logs at ./log/docker/node5_2026-03-03_04-33-51__f07d47f2-84fe-418f-b536-378d7a581a5d__wakuorg_nwaku:latest.log
2026-03-03 04:33:56.903 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 33111
2026-03-03 04:33:56.903 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:33:57.904 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33111/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:57.907 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:33:57.907 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:33:57.907 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33111/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:57.910 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.178.169/tcp/33112/p2p/16Uiu2HAmPjSEDtp3MGisGfk8HvarxCSatqNGA4z55u1jaHnduiEW","/ip4/172.18.178.169/tcp/33113/ws/p2p/16Uiu2HAmPjSEDtp3MGisGfk8HvarxCSatqNGA4z55u1jaHnduiEW"],"enrUri":"enr:-L24QBwW7bCNjDzrZonXONVWcQOmT42HHor3f24ByOZOPDaFBYjjBw_g9ZvDt3SPo6rmTxzVq24p0_EDs6sExlZd3YMCgmlkgnY0gmlwhKwSsqmKbXVsdGlhZGRyc5YACASsErKpBoFYAAoErBKyqQaBWd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDpJQj1Ak86vYWJXlQ9z_VbTyJP-DMmBUvhYGyASGJidODdGNwgoFYg3VkcIKBWoV3YWt1MgA"}'
2026-03-03 04:33:57.910 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:33:57.910 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:33111/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.123.131/tcp/63586/p2p/16Uiu2HAmFRj6JW5PnzAfxJ26amykxwwsoYj7uXihMpvkYumyv8Fh"]'
2026-03-03 04:33:57.947 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:33:57.948 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63585/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-03 04:33:57.954 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:33:57.957 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:50711/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "baea1cb5-8b19-4e4e-ab1f-50cff0c10abc", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-03 04:33:57.968 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"baea1cb5-8b19-4e4e-ab1f-50cff0c10abc","statusDesc":"OK"}'
2026-03-03 04:33:57.970 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:43805/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "3654abe2-1329-4153-a7a6-f66ada69ac89", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-03 04:33:57.982 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"3654abe2-1329-4153-a7a6-f66ada69ac89","statusDesc":"OK"}'
2026-03-03 04:33:57.982 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7138/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "6c7b1c90-4d04-419a-bb1c-c2072e4af2cd", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-03 04:33:57.991 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"6c7b1c90-4d04-419a-bb1c-c2072e4af2cd","statusDesc":"OK"}'
2026-03-03 04:33:57.992 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:33111/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "7d9ab315-b70a-4682-9395-fabcd0762dc5", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-03 04:33:58.001 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"7d9ab315-b70a-4682-9395-fabcd0762dc5","statusDesc":"OK"}'
2026-03-03 04:33:58.001 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63585/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "RmlsdGVyIHdvcmtzISE=", "contentTopic": "/test/1/waku-filter/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:33:58.018 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:33:58.018 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-03 04:33:58.119 DEBUG [src.steps.filter] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-03 04:33:58.119 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50711/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:58.122 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1772512438001785929,"ephemeral":false}]'
2026-03-03 04:33:58.124 DEBUG [src.steps.filter] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-03-03 04:33:58.124 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43805/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:58.127 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1772512438001785929,"ephemeral":false}]'
2026-03-03 04:33:58.128 DEBUG [src.steps.filter] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-03-03 04:33:58.128 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7138/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:58.131 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1772512438001785929,"ephemeral":false}]'
2026-03-03 04:33:58.132 DEBUG [src.steps.filter] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-03-03 04:33:58.132 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33111/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:33:58.135 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1772512438001785929,"ephemeral":false}]'
2026-03-03 04:33:58.137 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-03 04:33:58.138 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-03 04:33:58.139 DEBUG [src.node.waku_node] Stopping container with id c76fc5ef6294
2026-03-03 04:33:58.733 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:33:58.734 DEBUG [src.node.waku_node] Stopping container with id bd0ebd71ad09
2026-03-03 04:33:59.284 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:33:59.284 DEBUG [src.node.waku_node] Stopping container with id 3300d18e72fd
2026-03-03 04:33:59.547 ERROR [src.node.docker_mananger] Max retries reached for container c76fc5ef6294. Exiting log stream.
2026-03-03 04:33:59.866 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:33:59.866 DEBUG [src.node.waku_node] Stopping container with id e3ad47f76b7d
2026-03-03 04:34:00.098 ERROR [src.node.docker_mananger] Max retries reached for container bd0ebd71ad09. Exiting log stream.
2026-03-03 04:34:00.481 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:34:00.482 DEBUG [src.node.waku_node] Stopping container with id a7076fabbec1
2026-03-03 04:34:00.682 ERROR [src.node.docker_mananger] Max retries reached for container 3300d18e72fd. Exiting log stream.
2026-03-03 04:34:01.034 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:34:01.038 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-03 04:34:01.052 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-03 04:34:01.059 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-03 04:34:01.066 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-03 04:34:01.071 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-03 04:34:01.077 DEBUG [src.node.docker_mananger] No errors found in the waku logs.