82 lines
14 KiB
Plaintext

2026-03-13 04:38:40.468 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-13 04:38:40.469 DEBUG [tests.conftest] Running test: test_single_pubsub_topic[/waku/2/rs/2/0] with id: 2026-03-13_04-38-40__1e799a73-0648-4b4e-9672-cc419784a24d
2026-03-13 04:38:40.470 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-13 04:38:40.470 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-13 04:38:40.472 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-13 04:38:40.482 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-13 04:38:40.483 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-13_04-38-40__1e799a73-0648-4b4e-9672-cc419784a24d__wakuorg_nwaku:latest.log
2026-03-13 04:38:40.483 DEBUG [src.node.waku_node] Starting Node...
2026-03-13 04:38:40.483 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-13 04:38:40.487 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-13 04:38:40.488 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.159.208
2026-03-13 04:38:40.488 DEBUG [src.node.docker_mananger] Generated ports ['13288', '13289', '13290', '13291', '13292']
2026-03-13 04:38:40.489 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-13 04:38:40.490 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-13 04:38:40.490 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-13 04:38:40.490 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-13 04:38:40.490 DEBUG [src.node.waku_node] Using volumes []
2026-03-13 04:38:40.491 DEBUG [src.node.docker_mananger] docker run -i -t -p 13288:13288 -p 13289:13289 -p 13290:13290 -p 13291:13291 -p 13292:13292 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=13290 --rest-port=13288 --tcp-port=13289 --discv5-udp-port=13291 --rest-address=0.0.0.0 --nat=extip:172.18.159.208 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=7b7ab70ce9fdeb1be4a8dfc449698cdde59de5c4d8cdb84566ffa0022c5f5aff --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=13292 --metrics-logging=true --relay=true --filter=true
2026-03-13 04:38:40.664 ERROR [src.node.docker_mananger] Max retries reached for container 8beebab41ee8. Exiting log stream.
2026-03-13 04:38:40.701 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.159.208 waku b71e25e78ed92b2a89bc9f122da67e5d3b63f6acad2b206f46beddab86e882d4
2026-03-13 04:38:40.743 DEBUG [src.node.docker_mananger] Container started with ID b71e25e78ed9. Setting up logs at ./log/docker/node1_2026-03-13_04-38-40__1e799a73-0648-4b4e-9672-cc419784a24d__wakuorg_nwaku:latest.log
2026-03-13 04:38:40.743 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 13288
2026-03-13 04:38:40.743 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-13 04:38:41.383 ERROR [src.node.docker_mananger] Max retries reached for container 1d7ae5541e10. Exiting log stream.
2026-03-13 04:38:41.744 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13288/health" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:38:41.746 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-13 04:38:41.847 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13288/health" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:38:41.850 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-13 04:38:41.850 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-13 04:38:41.850 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13288/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:38:41.853 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.159.208/tcp/13289/p2p/16Uiu2HAmPoyxuNYxzXXdpiEULbeUw6UPs3xfQyUA2VmcpFFPAd7j","/ip4/172.18.159.208/tcp/13290/ws/p2p/16Uiu2HAmPoyxuNYxzXXdpiEULbeUw6UPs3xfQyUA2VmcpFFPAd7j"],"enrUri":"enr:-L24QKc_y5NOfzaWCXjEu578IgeLCwDWwyemGy1ZXeSXs_vGG-HO2XWwypwIt3E64LUuEKjnOD6LAt62XVxYd79ixW0CgmlkgnY0gmlwhKwSn9CKbXVsdGlhZGRyc5YACASsEp_QBjPpAAoErBKf0AYz6t0DgnJzhQACAQAAiXNlY3AyNTZrMaEDpb5Y2LnDTnxfnAV7b0ED2WRMpb48yH6TDuE6mJXu3B6DdGNwgjPpg3VkcIIz64V3YWt1MgU"}'
2026-03-13 04:38:41.853 INFO [src.node.waku_node] REST service is ready !!
2026-03-13 04:38:41.861 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-13 04:38:41.861 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-13_04-38-40__1e799a73-0648-4b4e-9672-cc419784a24d__wakuorg_nwaku:latest.log
2026-03-13 04:38:41.861 DEBUG [src.node.waku_node] Starting Node...
2026-03-13 04:38:41.861 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-13 04:38:41.863 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-13 04:38:41.863 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.231.96
2026-03-13 04:38:41.863 DEBUG [src.node.docker_mananger] Generated ports ['21153', '21154', '21155', '21156', '21157']
2026-03-13 04:38:41.863 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-13 04:38:41.863 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-13 04:38:41.863 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-13 04:38:41.864 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-13 04:38:41.864 DEBUG [src.node.waku_node] Using volumes []
2026-03-13 04:38:41.864 DEBUG [src.node.docker_mananger] docker run -i -t -p 21153:21153 -p 21154:21154 -p 21155:21155 -p 21156:21156 -p 21157:21157 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=21155 --rest-port=21153 --tcp-port=21154 --discv5-udp-port=21156 --rest-address=0.0.0.0 --nat=extip:172.18.231.96 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=2db2eceddd0d38e4e3d65e9e023acb142bef0cc0a7d3feab51c3e2dcea38cc09 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=21157 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QKc_y5NOfzaWCXjEu578IgeLCwDWwyemGy1ZXeSXs_vGG-HO2XWwypwIt3E64LUuEKjnOD6LAt62XVxYd79ixW0CgmlkgnY0gmlwhKwSn9CKbXVsdGlhZGRyc5YACASsEp_QBjPpAAoErBKf0AYz6t0DgnJzhQACAQAAiXNlY3AyNTZrMaEDpb5Y2LnDTnxfnAV7b0ED2WRMpb48yH6TDuE6mJXu3B6DdGNwgjPpg3VkcIIz64V3YWt1MgU
2026-03-13 04:38:42.062 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.231.96 waku 6d61471632de76f5ef1dce95be287c8603e2e30cd72a66ff250e25fc1fb16c27
2026-03-13 04:38:42.101 DEBUG [src.node.docker_mananger] Container started with ID 6d61471632de. Setting up logs at ./log/docker/node2_2026-03-13_04-38-40__1e799a73-0648-4b4e-9672-cc419784a24d__wakuorg_nwaku:latest.log
2026-03-13 04:38:42.101 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 21153
2026-03-13 04:38:42.101 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-13 04:38:43.101 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:21153/health" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:38:43.106 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-13 04:38:43.207 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:21153/health" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:38:43.209 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-13 04:38:43.210 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-13 04:38:43.210 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:21153/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:38:43.212 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.231.96/tcp/21154/p2p/16Uiu2HAm42s8BB425FqpBW3Z2z3ohFKQdpCJyCKSBa3ehKsnKzqd","/ip4/172.18.231.96/tcp/21155/ws/p2p/16Uiu2HAm42s8BB425FqpBW3Z2z3ohFKQdpCJyCKSBa3ehKsnKzqd"],"enrUri":"enr:-L24QF-bosNahtXa03Dx7073S2VsqhfbP5E4cQy7U0rxL5cQZW1vXRzRPhqSM3KO3FPjth_GlxbbpC6gJ9S43BhkOxYCgmlkgnY0gmlwhKwS52CKbXVsdGlhZGRyc5YACASsEudgBlKiAAoErBLnYAZSo90DgnJzhQACAQAAiXNlY3AyNTZrMaECf-FsxXSo4Zk36N7R5Pj9TGZjcJoaSf88CEMNszi8RWiDdGNwglKig3VkcIJSpIV3YWt1MgE"}'
2026-03-13 04:38:43.212 INFO [src.node.waku_node] REST service is ready !!
2026-03-13 04:38:43.213 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:21153/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.159.208/tcp/13289/p2p/16Uiu2HAmPoyxuNYxzXXdpiEULbeUw6UPs3xfQyUA2VmcpFFPAd7j"]'
2026-03-13 04:38:43.237 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:38:43.238 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13288/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-13 04:38:43.244 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:38:43.245 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:21153/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-13 04:38:43.250 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:38:43.251 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13288/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-13 04:38:43.263 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:38:43.263 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-13 04:38:43.364 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-13 04:38:43.364 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13288/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:38:43.366 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773376723251664799,"ephemeral":false,"proof":""}]'
2026-03-13 04:38:43.368 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-13 04:38:43.368 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:21153/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:38:43.370 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1773376723251664799,"ephemeral":false,"proof":""}]'
2026-03-13 04:38:43.373 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-13 04:38:43.374 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-13 04:38:43.374 DEBUG [src.node.waku_node] Stopping container with id b71e25e78ed9
2026-03-13 04:38:43.974 DEBUG [src.node.waku_node] Container stopped.
2026-03-13 04:38:43.974 DEBUG [src.node.waku_node] Stopping container with id 6d61471632de
2026-03-13 04:38:44.573 DEBUG [src.node.waku_node] Container stopped.
2026-03-13 04:38:44.574 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-13 04:38:44.611 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-13 04:38:44.636 DEBUG [src.node.docker_mananger] No errors found in the waku logs.