78 lines
13 KiB
Plaintext

2026-03-04 04:33:28.982 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-04 04:33:28.983 DEBUG [tests.conftest] Running test: test_2_nodes_different_cluster_different_shard with id: 2026-03-04_04-33-28__7ca2152e-c332-452e-9f22-f1139f5710f3
2026-03-04 04:33:28.983 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-04 04:33:28.983 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-04 04:33:28.983 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-04 04:33:28.990 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-04 04:33:28.990 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-04_04-33-28__7ca2152e-c332-452e-9f22-f1139f5710f3__wakuorg_nwaku:latest.log
2026-03-04 04:33:28.990 DEBUG [src.node.waku_node] Starting Node...
2026-03-04 04:33:28.990 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-04 04:33:28.992 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-04 04:33:28.992 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.109.113
2026-03-04 04:33:28.992 DEBUG [src.node.docker_mananger] Generated ports ['6517', '6518', '6519', '6520', '6521']
2026-03-04 04:33:28.992 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-04 04:33:28.992 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-04 04:33:28.992 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-04 04:33:28.992 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-04 04:33:28.992 DEBUG [src.node.waku_node] Using volumes []
2026-03-04 04:33:28.992 DEBUG [src.node.docker_mananger] docker run -i -t -p 6517:6517 -p 6518:6518 -p 6519:6519 -p 6520:6520 -p 6521:6521 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=6519 --rest-port=6517 --tcp-port=6518 --discv5-udp-port=6520 --rest-address=0.0.0.0 --nat=extip:172.18.109.113 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=e6f51de02d4eb6f0691d1dea1da6afdb0c96d9d6d4d056b6c57543d5aecec081 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=6521 --metrics-logging=true --relay=true --filter=true
2026-03-04 04:33:29.149 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.109.113 waku c004e0fc0617b13193ba5844d01def0a31883c2163642407cf58dc0d55f3eb94
2026-03-04 04:33:29.176 DEBUG [src.node.docker_mananger] Container started with ID c004e0fc0617. Setting up logs at ./log/docker/node1_2026-03-04_04-33-28__7ca2152e-c332-452e-9f22-f1139f5710f3__wakuorg_nwaku:latest.log
2026-03-04 04:33:29.177 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 6517
2026-03-04 04:33:29.177 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-04 04:33:29.305 ERROR [src.node.docker_mananger] Max retries reached for container d37486a5e07c. Exiting log stream.
2026-03-04 04:33:29.781 ERROR [src.node.docker_mananger] Max retries reached for container d476da50c54a. Exiting log stream.
2026-03-04 04:33:30.177 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6517/health" -H "Content-Type: application/json" -d 'None'
2026-03-04 04:33:30.180 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-04 04:33:30.180 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-04 04:33:30.180 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6517/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-04 04:33:30.183 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.109.113/tcp/6518/p2p/16Uiu2HAm5DCeo8hpiubNA8phBBDTW88y4qmzRs7A6H61YzEo3r3k","/ip4/172.18.109.113/tcp/6519/ws/p2p/16Uiu2HAm5DCeo8hpiubNA8phBBDTW88y4qmzRs7A6H61YzEo3r3k"],"enrUri":"enr:-L24QMAVu8bpgtXgNm7Faogcl3HdnogNisiJuNKarGFTY1vBF_k5Us1foYr0KlxrK60E6k3DE28G2iiRrXQ85fqWvSECgmlkgnY0gmlwhKwSbXGKbXVsdGlhZGRyc5YACASsEm1xBhl2AAoErBJtcQYZd90DgnJzhQACAQAAiXNlY3AyNTZrMaECkWMADY8BoPIe1zaXCAuVv1lfmh3cND9-3VIK1Sy7rpODdGNwghl2g3VkcIIZeIV3YWt1MgU"}'
2026-03-04 04:33:30.183 INFO [src.node.waku_node] REST service is ready !!
2026-03-04 04:33:30.189 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-04 04:33:30.189 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-04_04-33-28__7ca2152e-c332-452e-9f22-f1139f5710f3__wakuorg_nwaku:latest.log
2026-03-04 04:33:30.190 DEBUG [src.node.waku_node] Starting Node...
2026-03-04 04:33:30.190 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-04 04:33:30.191 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-04 04:33:30.191 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.29.127
2026-03-04 04:33:30.191 DEBUG [src.node.docker_mananger] Generated ports ['45745', '45746', '45747', '45748', '45749']
2026-03-04 04:33:30.191 DEBUG [src.node.waku_node] Cluster id was resolved to: 4
2026-03-04 04:33:30.192 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-04 04:33:30.192 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-04 04:33:30.192 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-04 04:33:30.192 DEBUG [src.node.waku_node] Using volumes []
2026-03-04 04:33:30.192 DEBUG [src.node.docker_mananger] docker run -i -t -p 45745:45745 -p 45746:45746 -p 45747:45747 -p 45748:45748 -p 45749:45749 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=45747 --rest-port=45745 --tcp-port=45746 --discv5-udp-port=45748 --rest-address=0.0.0.0 --nat=extip:172.18.29.127 --peer-exchange=true --discv5-discovery=true --cluster-id=4 --nodekey=08aaeed3bab97dd9b38ae42dffd0df156ac16d84c3444c4efe0dbec9459ece14 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=45749 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QMAVu8bpgtXgNm7Faogcl3HdnogNisiJuNKarGFTY1vBF_k5Us1foYr0KlxrK60E6k3DE28G2iiRrXQ85fqWvSECgmlkgnY0gmlwhKwSbXGKbXVsdGlhZGRyc5YACASsEm1xBhl2AAoErBJtcQYZd90DgnJzhQACAQAAiXNlY3AyNTZrMaECkWMADY8BoPIe1zaXCAuVv1lfmh3cND9-3VIK1Sy7rpODdGNwghl2g3VkcIIZeIV3YWt1MgU
2026-03-04 04:33:30.351 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.29.127 waku 853cc5521886aa0a030981bb263741647a0cbe6a03346e1b2a518184759795a0
2026-03-04 04:33:30.377 DEBUG [src.node.docker_mananger] Container started with ID 853cc5521886. Setting up logs at ./log/docker/node2_2026-03-04_04-33-28__7ca2152e-c332-452e-9f22-f1139f5710f3__wakuorg_nwaku:latest.log
2026-03-04 04:33:30.377 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 45745
2026-03-04 04:33:30.378 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-04 04:33:31.379 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:45745/health" -H "Content-Type: application/json" -d 'None'
2026-03-04 04:33:31.395 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-04 04:33:31.399 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-04 04:33:31.402 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:45745/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-04 04:33:31.407 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.29.127/tcp/45746/p2p/16Uiu2HAmH1MQNcj3UuykypHVUZqxRgxWZ37foPmsbVyYEQjDqiY6","/ip4/172.18.29.127/tcp/45747/ws/p2p/16Uiu2HAmH1MQNcj3UuykypHVUZqxRgxWZ37foPmsbVyYEQjDqiY6"],"enrUri":"enr:-L24QBc3tthyTuevZqRlY_miehWofdZiRjk3sauOMTtqkeCVNPDBo9m9cbdtnP0d8B8zO91x_v5-V7rNyMtSVLllBMwCgmlkgnY0gmlwhKwSHX-KbXVsdGlhZGRyc5YACASsEh1_BrKyAAoErBIdfways90DgnJzhQAEAQAAiXNlY3AyNTZrMaEDQKYrNQKRgJg9wbnsor8SG98N0hTUkFFnk4pRavtemY-DdGNwgrKyg3VkcIKytIV3YWt1MgE"}'
2026-03-04 04:33:31.407 INFO [src.node.waku_node] REST service is ready !!
2026-03-04 04:33:31.407 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:45745/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.109.113/tcp/6518/p2p/16Uiu2HAm5DCeo8hpiubNA8phBBDTW88y4qmzRs7A6H61YzEo3r3k"]'
2026-03-04 04:33:31.427 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-04 04:33:31.429 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6517/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-04 04:33:31.438 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-04 04:33:31.439 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:45745/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/4/1"]'
2026-03-04 04:33:31.445 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-04 04:33:31.446 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6517/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-04 04:33:31.450 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-04 04:33:31.450 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-04 04:33:31.550 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-04 04:33:31.551 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6517/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-04 04:33:31.553 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772598811446161382,"ephemeral":false,"proof":""}]'
2026-03-04 04:33:31.555 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-04 04:33:31.555 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:45745/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-04 04:33:31.558 ERROR [src.node.api_clients.base_client] HTTP error occurred: 404 Client Error: Not Found for url: http://127.0.0.1:45745/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0. Response content: b''
2026-03-04 04:33:31.560 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-04 04:33:31.561 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-04 04:33:31.561 DEBUG [src.node.waku_node] Stopping container with id c004e0fc0617
2026-03-04 04:33:32.015 DEBUG [src.node.waku_node] Container stopped.
2026-03-04 04:33:32.015 DEBUG [src.node.waku_node] Stopping container with id 853cc5521886
2026-03-04 04:33:32.499 DEBUG [src.node.waku_node] Container stopped.
2026-03-04 04:33:32.501 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-04 04:33:32.509 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-04 04:33:32.515 DEBUG [src.node.docker_mananger] No errors found in the waku logs.