180 lines
32 KiB
Plaintext

2026-02-01 04:51:40.816 DEBUG [tests.conftest] Running fixture setup: test_id
2026-02-01 04:51:40.816 DEBUG [tests.conftest] Running test: test_static_shard_relay with id: 2026-02-01_04-51-40__bc8519c1-b6d0-4517-acf4-c3b709dec3d6
2026-02-01 04:51:40.817 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-02-01 04:51:40.817 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-02-01 04:51:40.817 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-02-01 04:51:40.825 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:51:40.825 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-02-01_04-51-40__bc8519c1-b6d0-4517-acf4-c3b709dec3d6__wakuorg_nwaku:latest.log
2026-02-01 04:51:40.825 DEBUG [src.node.waku_node] Starting Node...
2026-02-01 04:51:40.825 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-01 04:51:40.827 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-01 04:51:40.827 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.152.26
2026-02-01 04:51:40.827 DEBUG [src.node.docker_mananger] Generated ports ['9357', '9358', '9359', '9360', '9361']
2026-02-01 04:51:40.827 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-02-01 04:51:40.827 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-02-01 04:51:40.828 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-01 04:51:40.828 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-01 04:51:40.828 DEBUG [src.node.waku_node] Using volumes []
2026-02-01 04:51:40.828 DEBUG [src.node.docker_mananger] docker run -i -t -p 9357:9357 -p 9358:9358 -p 9359:9359 -p 9360:9360 -p 9361:9361 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=9359 --rest-port=9357 --tcp-port=9358 --discv5-udp-port=9360 --rest-address=0.0.0.0 --nat=extip:172.18.152.26 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=e0fa6ddc1f682efffcce4e6bb0af3bd8239f4df86b6ecaea8c9f4f67a36070d3 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=9361 --metrics-logging=true --relay=true --filter=true
2026-02-01 04:51:41.031 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.152.26 waku 6a8ee7e30d0bfc2ddbf5ae58d92ace673b750175ae74ca2ee882c7d583b372b4
2026-02-01 04:51:41.055 ERROR [src.node.docker_mananger] Max retries reached for container ea8c234f7083. Exiting log stream.
2026-02-01 04:51:41.064 DEBUG [src.node.docker_mananger] Container started with ID 6a8ee7e30d0b. Setting up logs at ./log/docker/node1_2026-02-01_04-51-40__bc8519c1-b6d0-4517-acf4-c3b709dec3d6__wakuorg_nwaku:latest.log
2026-02-01 04:51:41.064 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 9357
2026-02-01 04:51:41.065 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-01 04:51:41.614 ERROR [src.node.docker_mananger] Max retries reached for container cbc913e2377f. Exiting log stream.
2026-02-01 04:51:42.065 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9357/health" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:42.069 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-01 04:51:42.069 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-01 04:51:42.069 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9357/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:42.072 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.152.26/tcp/9358/p2p/16Uiu2HAmMNLa7kZPAtugNDFFKLDdonMMkPiDtpoYxRUi3jw7JdJf","/ip4/172.18.152.26/tcp/9359/ws/p2p/16Uiu2HAmMNLa7kZPAtugNDFFKLDdonMMkPiDtpoYxRUi3jw7JdJf"],"enrUri":"enr:-L24QLSg0op_X_k4iWJBIVJFdV63UoqLQRMmcmVig_rOQTb-eiZ2KgD5MYZ19iQRtwGgtcH6KdwxlKkYQ2v6b8mpZs4CgmlkgnY0gmlwhKwSmBqKbXVsdGlhZGRyc5YACASsEpgaBiSOAAoErBKYGgYkj90DgnJzhQACAQAAiXNlY3AyNTZrMaEDgXUsN4rgL3NgyjE7XtQGjOdUxBNA8QshNiajsPukAliDdGNwgiSOg3VkcIIkkIV3YWt1MgU"}'
2026-02-01 04:51:42.072 INFO [src.node.waku_node] REST service is ready !!
2026-02-01 04:51:42.079 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:51:42.079 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-02-01_04-51-40__bc8519c1-b6d0-4517-acf4-c3b709dec3d6__wakuorg_nwaku:latest.log
2026-02-01 04:51:42.079 DEBUG [src.node.waku_node] Starting Node...
2026-02-01 04:51:42.079 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-01 04:51:42.080 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-01 04:51:42.081 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.57.248
2026-02-01 04:51:42.081 DEBUG [src.node.docker_mananger] Generated ports ['23667', '23668', '23669', '23670', '23671']
2026-02-01 04:51:42.081 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-02-01 04:51:42.081 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-02-01 04:51:42.081 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-01 04:51:42.081 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-01 04:51:42.081 DEBUG [src.node.waku_node] Using volumes []
2026-02-01 04:51:42.081 DEBUG [src.node.docker_mananger] docker run -i -t -p 23667:23667 -p 23668:23668 -p 23669:23669 -p 23670:23670 -p 23671:23671 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=23669 --rest-port=23667 --tcp-port=23668 --discv5-udp-port=23670 --rest-address=0.0.0.0 --nat=extip:172.18.57.248 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=a6aefedccf0c5abeff1c44befa1aac07c68d4c8dee4fbfc9ae0a2ccbee7e3b96 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=23671 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QLSg0op_X_k4iWJBIVJFdV63UoqLQRMmcmVig_rOQTb-eiZ2KgD5MYZ19iQRtwGgtcH6KdwxlKkYQ2v6b8mpZs4CgmlkgnY0gmlwhKwSmBqKbXVsdGlhZGRyc5YACASsEpgaBiSOAAoErBKYGgYkj90DgnJzhQACAQAAiXNlY3AyNTZrMaEDgXUsN4rgL3NgyjE7XtQGjOdUxBNA8QshNiajsPukAliDdGNwgiSOg3VkcIIkkIV3YWt1MgU
2026-02-01 04:51:42.288 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.57.248 waku e623b7bf0d9d200f86340a557b3561e7d466ff70678f1aaff721af5d29297b9e
2026-02-01 04:51:42.325 DEBUG [src.node.docker_mananger] Container started with ID e623b7bf0d9d. Setting up logs at ./log/docker/node2_2026-02-01_04-51-40__bc8519c1-b6d0-4517-acf4-c3b709dec3d6__wakuorg_nwaku:latest.log
2026-02-01 04:51:42.325 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 23667
2026-02-01 04:51:42.325 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-01 04:51:43.326 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23667/health" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:43.334 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"}]}'
2026-02-01 04:51:43.335 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-01 04:51:43.335 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23667/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:43.340 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.57.248/tcp/23668/p2p/16Uiu2HAkuZtabKvxfWXCnoF7BTa6m81CiaewViPp4R6sFwMFLPSZ","/ip4/172.18.57.248/tcp/23669/ws/p2p/16Uiu2HAkuZtabKvxfWXCnoF7BTa6m81CiaewViPp4R6sFwMFLPSZ"],"enrUri":"enr:-L24QA0qyzFnADdN_5I9cib3g6G-nLXa-JoHKErjX7aYkZcPNEBtiseOmy9hgiaeB1QQsxgvUe6nAfbJgCvA7LmXm9oCgmlkgnY0gmlwhKwSOfiKbXVsdGlhZGRyc5YACASsEjn4Blx0AAoErBI5-AZcdd0DgnJzhQACAQAAiXNlY3AyNTZrMaECAhrksLGM3V_wpnt39mUkscOKbLlUstJZHpxEbD6hX7qDdGNwglx0g3VkcIJcdoV3YWt1MgE"}'
2026-02-01 04:51:43.340 INFO [src.node.waku_node] REST service is ready !!
2026-02-01 04:51:43.341 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23667/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.152.26/tcp/9358/p2p/16Uiu2HAmMNLa7kZPAtugNDFFKLDdonMMkPiDtpoYxRUi3jw7JdJf"]'
2026-02-01 04:51:43.345 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:51:43.352 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:51:43.352 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-02-01_04-51-40__bc8519c1-b6d0-4517-acf4-c3b709dec3d6__wakuorg_nwaku:latest.log
2026-02-01 04:51:43.352 DEBUG [src.node.waku_node] Starting Node...
2026-02-01 04:51:43.353 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-01 04:51:43.354 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-01 04:51:43.354 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.145.131
2026-02-01 04:51:43.354 DEBUG [src.node.docker_mananger] Generated ports ['19203', '19204', '19205', '19206', '19207']
2026-02-01 04:51:43.354 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-02-01 04:51:43.355 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-02-01 04:51:43.355 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-01 04:51:43.355 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-01 04:51:43.355 DEBUG [src.node.waku_node] Using volumes []
2026-02-01 04:51:43.355 DEBUG [src.node.docker_mananger] docker run -i -t -p 19203:19203 -p 19204:19204 -p 19205:19205 -p 19206:19206 -p 19207:19207 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=19205 --rest-port=19203 --tcp-port=19204 --discv5-udp-port=19206 --rest-address=0.0.0.0 --nat=extip:172.18.145.131 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=223afcee0dafe43fba2c5327acc4c8e1db3bc52fc40b4b7b42ceb49d8ddd40fa --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=19207 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QLSg0op_X_k4iWJBIVJFdV63UoqLQRMmcmVig_rOQTb-eiZ2KgD5MYZ19iQRtwGgtcH6KdwxlKkYQ2v6b8mpZs4CgmlkgnY0gmlwhKwSmBqKbXVsdGlhZGRyc5YACASsEpgaBiSOAAoErBKYGgYkj90DgnJzhQACAQAAiXNlY3AyNTZrMaEDgXUsN4rgL3NgyjE7XtQGjOdUxBNA8QshNiajsPukAliDdGNwgiSOg3VkcIIkkIV3YWt1MgU
2026-02-01 04:51:43.545 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.145.131 waku 7f8d125d3349c6602f2f77d3268508e36849db1c6310af9310598c4331d4b935
2026-02-01 04:51:43.577 DEBUG [src.node.docker_mananger] Container started with ID 7f8d125d3349. Setting up logs at ./log/docker/node3_2026-02-01_04-51-40__bc8519c1-b6d0-4517-acf4-c3b709dec3d6__wakuorg_nwaku:latest.log
2026-02-01 04:51:43.578 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 19203
2026-02-01 04:51:43.579 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-01 04:51:44.581 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19203/health" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:44.590 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"READY"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"}]}'
2026-02-01 04:51:44.592 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-01 04:51:44.593 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19203/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:44.601 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.145.131/tcp/19204/p2p/16Uiu2HAm28YFikQuA9r7gSsrPYh4uaSvbiAMXuqwyvTwJysJU1C4","/ip4/172.18.145.131/tcp/19205/ws/p2p/16Uiu2HAm28YFikQuA9r7gSsrPYh4uaSvbiAMXuqwyvTwJysJU1C4"],"enrUri":"enr:-L24QG74yXDsiW-P2dZc8NS7ky0i9GZXYG4WZIkL8cMQj_PBKwj5RwsqF8djKm4ZCSPy-iiqAWsG2-m_7AaAkIXa-O4CgmlkgnY0gmlwhKwSkYOKbXVsdGlhZGRyc5YACASsEpGDBksEAAoErBKRgwZLBd0DgnJzhQACAQAAiXNlY3AyNTZrMaECY54zGOHtTmGqT7aHkL5Czxsy_HExNRvYK6crjwI5eomDdGNwgksEg3VkcIJLBoV3YWt1MgE"}'
2026-02-01 04:51:44.602 INFO [src.node.waku_node] REST service is ready !!
2026-02-01 04:51:44.603 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19203/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.152.26/tcp/9358/p2p/16Uiu2HAmMNLa7kZPAtugNDFFKLDdonMMkPiDtpoYxRUi3jw7JdJf"]'
2026-02-01 04:51:44.606 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:51:44.614 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:51:44.614 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-02-01_04-51-40__bc8519c1-b6d0-4517-acf4-c3b709dec3d6__wakuorg_nwaku:latest.log
2026-02-01 04:51:44.614 DEBUG [src.node.waku_node] Starting Node...
2026-02-01 04:51:44.614 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-01 04:51:44.615 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-01 04:51:44.616 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.187.241
2026-02-01 04:51:44.616 DEBUG [src.node.docker_mananger] Generated ports ['47434', '47435', '47436', '47437', '47438']
2026-02-01 04:51:44.616 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-02-01 04:51:44.616 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-02-01 04:51:44.616 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-01 04:51:44.616 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-01 04:51:44.616 DEBUG [src.node.waku_node] Using volumes []
2026-02-01 04:51:44.616 DEBUG [src.node.docker_mananger] docker run -i -t -p 47434:47434 -p 47435:47435 -p 47436:47436 -p 47437:47437 -p 47438:47438 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=47436 --rest-port=47434 --tcp-port=47435 --discv5-udp-port=47437 --rest-address=0.0.0.0 --nat=extip:172.18.187.241 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=b5422bcdfbfe1e13c872bb1c9f4ba8cdcfbdbc5dcdffccc5e7dd90c3bcbc6cf6 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=47438 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QLSg0op_X_k4iWJBIVJFdV63UoqLQRMmcmVig_rOQTb-eiZ2KgD5MYZ19iQRtwGgtcH6KdwxlKkYQ2v6b8mpZs4CgmlkgnY0gmlwhKwSmBqKbXVsdGlhZGRyc5YACASsEpgaBiSOAAoErBKYGgYkj90DgnJzhQACAQAAiXNlY3AyNTZrMaEDgXUsN4rgL3NgyjE7XtQGjOdUxBNA8QshNiajsPukAliDdGNwgiSOg3VkcIIkkIV3YWt1MgU
2026-02-01 04:51:44.806 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.187.241 waku fedf01b1c080dc348c91b2112ae1d7c8a6208ec8631f1c95c11069243553c421
2026-02-01 04:51:44.841 DEBUG [src.node.docker_mananger] Container started with ID fedf01b1c080. Setting up logs at ./log/docker/node4_2026-02-01_04-51-40__bc8519c1-b6d0-4517-acf4-c3b709dec3d6__wakuorg_nwaku:latest.log
2026-02-01 04:51:44.842 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 47434
2026-02-01 04:51:44.842 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-01 04:51:45.842 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:47434/health" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:45.852 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"READY"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"}]}'
2026-02-01 04:51:45.852 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-01 04:51:45.853 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:47434/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:45.855 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.187.241/tcp/47435/p2p/16Uiu2HAmNExLjAc4khy3xMrWf2aqtAUYN74u1C9QqSM5nqTQ6Wf5","/ip4/172.18.187.241/tcp/47436/ws/p2p/16Uiu2HAmNExLjAc4khy3xMrWf2aqtAUYN74u1C9QqSM5nqTQ6Wf5"],"enrUri":"enr:-L24QC59Oo02Qth0Tm0sIdzGg_whCTOOfaMn0O8ipKzxvjNbHpjrL_eDhqEr0UuH-99oTrWHAVf0L225c04KwojMipkCgmlkgnY0gmlwhKwSu_GKbXVsdGlhZGRyc5YACASsErvxBrlLAAoErBK78Qa5TN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDjmyoUK41SzvaLOeJhD1HQUNcuVE0pHI6EXzmb4CNS6yDdGNwgrlLg3VkcIK5TYV3YWt1MgE"}'
2026-02-01 04:51:45.856 INFO [src.node.waku_node] REST service is ready !!
2026-02-01 04:51:45.856 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:47434/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.152.26/tcp/9358/p2p/16Uiu2HAmMNLa7kZPAtugNDFFKLDdonMMkPiDtpoYxRUi3jw7JdJf"]'
2026-02-01 04:51:45.858 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:51:45.865 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-01 04:51:45.865 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-02-01_04-51-40__bc8519c1-b6d0-4517-acf4-c3b709dec3d6__wakuorg_nwaku:latest.log
2026-02-01 04:51:45.865 DEBUG [src.node.waku_node] Starting Node...
2026-02-01 04:51:45.866 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-01 04:51:45.867 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-01 04:51:45.867 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.231.24
2026-02-01 04:51:45.867 DEBUG [src.node.docker_mananger] Generated ports ['1089', '1090', '1091', '1092', '1093']
2026-02-01 04:51:45.867 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-02-01 04:51:45.867 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-02-01 04:51:45.867 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-01 04:51:45.868 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-01 04:51:45.868 DEBUG [src.node.waku_node] Using volumes []
2026-02-01 04:51:45.868 DEBUG [src.node.docker_mananger] docker run -i -t -p 1089:1089 -p 1090:1090 -p 1091:1091 -p 1092:1092 -p 1093:1093 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=1091 --rest-port=1089 --tcp-port=1090 --discv5-udp-port=1092 --rest-address=0.0.0.0 --nat=extip:172.18.231.24 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=1020bb71ca6eae92129b739bc632fa07fccb9d6dbce64f5edbbbf18a04cc98d0 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=1093 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QLSg0op_X_k4iWJBIVJFdV63UoqLQRMmcmVig_rOQTb-eiZ2KgD5MYZ19iQRtwGgtcH6KdwxlKkYQ2v6b8mpZs4CgmlkgnY0gmlwhKwSmBqKbXVsdGlhZGRyc5YACASsEpgaBiSOAAoErBKYGgYkj90DgnJzhQACAQAAiXNlY3AyNTZrMaEDgXUsN4rgL3NgyjE7XtQGjOdUxBNA8QshNiajsPukAliDdGNwgiSOg3VkcIIkkIV3YWt1MgU
2026-02-01 04:51:46.064 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.231.24 waku 4d50e42fa1a731b30a5de0fe9253bf121cb8601fd4a23cc61c4f92152ec9b6d8
2026-02-01 04:51:46.100 DEBUG [src.node.docker_mananger] Container started with ID 4d50e42fa1a7. Setting up logs at ./log/docker/node5_2026-02-01_04-51-40__bc8519c1-b6d0-4517-acf4-c3b709dec3d6__wakuorg_nwaku:latest.log
2026-02-01 04:51:46.101 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 1089
2026-02-01 04:51:46.101 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-01 04:51:47.103 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1089/health" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:47.110 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"READY"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"}]}'
2026-02-01 04:51:47.111 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-01 04:51:47.111 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1089/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:47.115 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.231.24/tcp/1090/p2p/16Uiu2HAm2d1ddo95FjaKwUNGwGXBN4wfbwayp48QmT39B3wnShfE","/ip4/172.18.231.24/tcp/1091/ws/p2p/16Uiu2HAm2d1ddo95FjaKwUNGwGXBN4wfbwayp48QmT39B3wnShfE"],"enrUri":"enr:-L24QIFZDDwPDs8xmzSjYoDU3S8TuY8lzf35x8nVF94kEWS1DfORm0SJ0ZRGXpZF3yxrhKkZjOCeAFuhN4vbE3Si9n0CgmlkgnY0gmlwhKwS5xiKbXVsdGlhZGRyc5YACASsEucYBgRCAAoErBLnGAYEQ90DgnJzhQACAQAAiXNlY3AyNTZrMaECaulrIQXi0YCdWrgDQhHCjXBSe1cP8H7U906MDQKWyUGDdGNwggRCg3VkcIIERIV3YWt1MgE"}'
2026-02-01 04:51:47.116 INFO [src.node.waku_node] REST service is ready !!
2026-02-01 04:51:47.116 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:1089/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.152.26/tcp/9358/p2p/16Uiu2HAmMNLa7kZPAtugNDFFKLDdonMMkPiDtpoYxRUi3jw7JdJf"]'
2026-02-01 04:51:47.119 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:51:47.119 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:9357/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-02-01 04:51:47.122 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:51:47.122 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23667/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-02-01 04:51:47.125 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:51:47.125 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19203/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-02-01 04:51:47.127 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:51:47.128 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:47434/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-02-01 04:51:47.130 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:51:47.131 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:1089/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-02-01 04:51:47.133 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:51:47.134 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:9357/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-02-01 04:51:47.140 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-01 04:51:47.142 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-02-01 04:51:47.243 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-02-01 04:51:47.243 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9357/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:47.245 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1769921507133888594,"ephemeral":false,"proof":""}]'
2026-02-01 04:51:47.246 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-02-01 04:51:47.247 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23667/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:47.249 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1769921507133888594,"ephemeral":false,"proof":""}]'
2026-02-01 04:51:47.250 DEBUG [src.steps.sharding] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-02-01 04:51:47.251 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19203/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:47.253 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1769921507133888594,"ephemeral":false,"proof":""}]'
2026-02-01 04:51:47.254 DEBUG [src.steps.sharding] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-02-01 04:51:47.254 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:47434/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:47.256 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1769921507133888594,"ephemeral":false,"proof":""}]'
2026-02-01 04:51:47.258 DEBUG [src.steps.sharding] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-02-01 04:51:47.258 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1089/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-01 04:51:47.260 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1769921507133888594,"ephemeral":false,"proof":""}]'
2026-02-01 04:51:47.263 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-02-01 04:51:47.264 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-02-01 04:51:47.264 DEBUG [src.node.waku_node] Stopping container with id 6a8ee7e30d0b
2026-02-01 04:51:47.880 DEBUG [src.node.waku_node] Container stopped.
2026-02-01 04:51:47.880 DEBUG [src.node.waku_node] Stopping container with id e623b7bf0d9d
2026-02-01 04:51:48.405 DEBUG [src.node.waku_node] Container stopped.
2026-02-01 04:51:48.407 DEBUG [src.node.waku_node] Stopping container with id 7f8d125d3349
2026-02-01 04:51:48.693 ERROR [src.node.docker_mananger] Max retries reached for container 6a8ee7e30d0b. Exiting log stream.
2026-02-01 04:51:48.978 DEBUG [src.node.waku_node] Container stopped.
2026-02-01 04:51:48.979 DEBUG [src.node.waku_node] Stopping container with id fedf01b1c080
2026-02-01 04:51:49.240 ERROR [src.node.docker_mananger] Max retries reached for container e623b7bf0d9d. Exiting log stream.
2026-02-01 04:51:49.531 DEBUG [src.node.waku_node] Container stopped.
2026-02-01 04:51:49.533 DEBUG [src.node.waku_node] Stopping container with id 4d50e42fa1a7
2026-02-01 04:51:49.811 ERROR [src.node.docker_mananger] Max retries reached for container 7f8d125d3349. Exiting log stream.
2026-02-01 04:51:50.124 DEBUG [src.node.waku_node] Container stopped.
2026-02-01 04:51:50.127 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-02-01 04:51:50.141 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-01 04:51:50.150 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-01 04:51:50.159 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-01 04:51:50.168 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-01 04:51:50.174 DEBUG [src.node.docker_mananger] No errors found in the waku logs.