2026-03-31 02:44:07 +00:00

110 lines
18 KiB
Plaintext

2026-03-31 02:26:07.735 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-31 02:26:07.736 DEBUG [tests.conftest] Running test: test_sync_flags_node2_start_later with id: 2026-03-31_02-26-07__8210d528-025c-4bdb-b566-455007b4e92b
2026-03-31 02:26:07.736 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-31 02:26:07.743 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-31 02:26:07.743 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-31_02-26-07__8210d528-025c-4bdb-b566-455007b4e92b__wakuorg_nwaku:latest.log
2026-03-31 02:26:07.749 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-31 02:26:07.749 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-31_02-26-07__8210d528-025c-4bdb-b566-455007b4e92b__wakuorg_nwaku:latest.log
2026-03-31 02:26:07.755 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-31 02:26:07.755 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-31_02-26-07__8210d528-025c-4bdb-b566-455007b4e92b__wakuorg_nwaku:latest.log
2026-03-31 02:26:07.755 DEBUG [src.steps.store] Running fixture setup: store_setup
2026-03-31 02:26:07.756 DEBUG [src.node.waku_node] Starting Node...
2026-03-31 02:26:07.756 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-31 02:26:07.758 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-31 02:26:07.758 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.244.98
2026-03-31 02:26:07.758 DEBUG [src.node.docker_mananger] Generated ports ['6311', '6312', '6313', '6314', '6315']
2026-03-31 02:26:07.758 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-31 02:26:07.758 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-31 02:26:07.759 DEBUG [src.node.waku_node] Using volumes []
2026-03-31 02:26:07.759 DEBUG [src.node.docker_mananger] docker run -i -t -p 6311:6311 -p 6312:6312 -p 6313:6313 -p 6314:6314 -p 6315:6315 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=6313 --rest-port=6311 --tcp-port=6312 --discv5-udp-port=6314 --rest-address=0.0.0.0 --nat=extip:172.18.244.98 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=cdcc1c8fa90abce3cfab71bf6fc8d46e1b9dbe9daf1a9bdff8c9c98d1f1ea95c --store-sync=true --store=true --store-sync-range=45 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=6315 --metrics-logging=true --store-sync-relay-jitter=0 --relay=true
2026-03-31 02:26:07.948 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.244.98 waku 1127a29c266336460dd29fbb05bca2b50106ec839a8dd01edf2da8036bf1924f
2026-03-31 02:26:07.985 DEBUG [src.node.docker_mananger] Container started with ID 1127a29c2663. Setting up logs at ./log/docker/node1_2026-03-31_02-26-07__8210d528-025c-4bdb-b566-455007b4e92b__wakuorg_nwaku:latest.log
2026-03-31 02:26:07.986 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 6311
2026-03-31 02:26:07.987 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-31 02:26:08.075 ERROR [src.node.docker_mananger] Max retries reached for container d511ac6a82dc. Exiting log stream.
2026-03-31 02:26:08.523 ERROR [src.node.docker_mananger] Max retries reached for container 3aefb77d5c3b. Exiting log stream.
2026-03-31 02:26:08.988 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6311/health" -H "Content-Type: application/json" -d 'None'
2026-03-31 02:26:08.991 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-31 02:26:08.992 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-31 02:26:08.992 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6311/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-31 02:26:08.994 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.244.98/tcp/6312/p2p/16Uiu2HAmKq5rHobmVsKnKo6DSYWKTuhNeschSnGSmKTkHNu3uWok","/ip4/172.18.244.98/tcp/6313/ws/p2p/16Uiu2HAmKq5rHobmVsKnKo6DSYWKTuhNeschSnGSmKTkHNu3uWok"],"enrUri":"enr:-L24QHdQJXPA0R2vlYoLLndqFVu6SyFX1ocEG9bHGw6KT8BNCTHbX4ffFb-3p0X5aAvyMY7ON8lek6pC1jwUeATdRaICgmlkgnY0gmlwhKwS9GKKbXVsdGlhZGRyc5YACASsEvRiBhioAAoErBL0YgYYqd0DgnJzhQDGAQAAiXNlY3AyNTZrMaEDapfV4JAdfuf1JoixZeJTivQNF2kyBId8raOiUoNw6KuDdGNwghiog3VkcIIYqoV3YWt1MhM"}'
2026-03-31 02:26:08.994 INFO [src.node.waku_node] REST service is ready !!
2026-03-31 02:26:08.994 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6311/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/0"]'
2026-03-31 02:26:09.009 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-31 02:26:09.010 DEBUG [src.steps.store] Relaying message
2026-03-31 02:26:09.010 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-31 02:26:09.014 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-31 02:26:09.014 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-31 02:26:09.215 DEBUG [src.steps.store] Relaying message
2026-03-31 02:26:09.215 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-31 02:26:09.219 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-31 02:26:09.220 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-31 02:26:09.420 DEBUG [src.steps.store] Relaying message
2026-03-31 02:26:09.421 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-31 02:26:09.425 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-31 02:26:09.425 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-31 02:26:09.626 DEBUG [src.steps.store] Relaying message
2026-03-31 02:26:09.626 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-31 02:26:09.630 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-31 02:26:09.631 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-31 02:26:09.831 DEBUG [src.steps.store] Relaying message
2026-03-31 02:26:09.832 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-31 02:26:09.836 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-31 02:26:09.836 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-31 02:26:10.037 DEBUG [src.steps.store] Relaying message
2026-03-31 02:26:10.037 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-31 02:26:10.041 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-31 02:26:10.042 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-31 02:26:10.242 DEBUG [src.steps.store] Relaying message
2026-03-31 02:26:10.243 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-31 02:26:10.247 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-31 02:26:10.247 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-31 02:26:10.448 DEBUG [src.steps.store] Relaying message
2026-03-31 02:26:10.448 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-31 02:26:10.452 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-31 02:26:10.453 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-31 02:26:10.653 DEBUG [src.steps.store] Relaying message
2026-03-31 02:26:10.654 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-31 02:26:10.658 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-31 02:26:10.658 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-31 02:26:10.859 DEBUG [src.steps.store] Relaying message
2026-03-31 02:26:10.859 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6311/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-31 02:26:10.863 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-31 02:26:10.863 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-31 02:26:11.064 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-31 02:26:12.064 DEBUG [src.node.waku_node] Starting Node...
2026-03-31 02:26:12.064 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-31 02:26:12.066 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-31 02:26:12.067 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.32.13
2026-03-31 02:26:12.067 DEBUG [src.node.docker_mananger] Generated ports ['21889', '21890', '21891', '21892', '21893']
2026-03-31 02:26:12.067 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-31 02:26:12.067 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-31 02:26:12.067 DEBUG [src.node.waku_node] Using volumes []
2026-03-31 02:26:12.067 DEBUG [src.node.docker_mananger] docker run -i -t -p 21889:21889 -p 21890:21890 -p 21891:21891 -p 21892:21892 -p 21893:21893 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=21891 --rest-port=21889 --tcp-port=21890 --discv5-udp-port=21892 --rest-address=0.0.0.0 --nat=extip:172.18.32.13 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=ebbdcbf3ffa0a84dc71eaaf1c6ee94defa6100da1cc6dbbcb2550fa6c8c32a1c --store-sync=true --store=true --store-sync-range=45 --store-sync-interval=10 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=21893 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --discv5-bootstrap-node=enr:-L24QHdQJXPA0R2vlYoLLndqFVu6SyFX1ocEG9bHGw6KT8BNCTHbX4ffFb-3p0X5aAvyMY7ON8lek6pC1jwUeATdRaICgmlkgnY0gmlwhKwS9GKKbXVsdGlhZGRyc5YACASsEvRiBhioAAoErBL0YgYYqd0DgnJzhQDGAQAAiXNlY3AyNTZrMaEDapfV4JAdfuf1JoixZeJTivQNF2kyBId8raOiUoNw6KuDdGNwghiog3VkcIIYqoV3YWt1MhM
2026-03-31 02:26:12.275 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.32.13 waku dcee17e51abc5c79bcb5266383acd1d63cace5a493491d103416c418a53d4022
2026-03-31 02:26:12.313 DEBUG [src.node.docker_mananger] Container started with ID dcee17e51abc. Setting up logs at ./log/docker/node2_2026-03-31_02-26-07__8210d528-025c-4bdb-b566-455007b4e92b__wakuorg_nwaku:latest.log
2026-03-31 02:26:12.314 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 21889
2026-03-31 02:26:12.314 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-31 02:26:13.315 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:21889/health" -H "Content-Type: application/json" -d 'None'
2026-03-31 02:26:13.318 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-31 02:26:13.318 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-31 02:26:13.318 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:21889/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-31 02:26:13.320 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.32.13/tcp/21890/p2p/16Uiu2HAmG1EMXS542fRnACokbr4ZVkYbpTVAVZZAyd9htn2pbNpP","/ip4/172.18.32.13/tcp/21891/ws/p2p/16Uiu2HAmG1EMXS542fRnACokbr4ZVkYbpTVAVZZAyd9htn2pbNpP"],"enrUri":"enr:-L24QCfxrBKj3cQkYe-CfOMjFDcz_R2YklychTNdCVoNclcPSQDpSe1XFy6A4uQ8EataBsQgEpvPlBav9edK32rqEKYCgmlkgnY0gmlwhKwSIA2KbXVsdGlhZGRyc5YACASsEiANBlWCAAoErBIgDQZVg90DgnJzhQDGAQAAiXNlY3AyNTZrMaEDMcKC62rgpgYkzxJILP3cRsZhoRn72RxuEz5I6EozXFCDdGNwglWCg3VkcIJVhIV3YWt1MhI"}'
2026-03-31 02:26:13.321 INFO [src.node.waku_node] REST service is ready !!
2026-03-31 02:26:13.321 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:21889/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.244.98/tcp/6312/p2p/16Uiu2HAmKq5rHobmVsKnKo6DSYWKTuhNeschSnGSmKTkHNu3uWok"]'
2026-03-31 02:26:13.323 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-31 02:26:13.324 DEBUG [src.libs.common] Sleeping for 65 seconds
2026-03-31 02:27:18.324 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-31 02:27:18.325 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:21889/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F198%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-31 02:27:18.328 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x576e6c9d8b832abb92322da409cb88b6c497a92210d9d815febc53213125cfcf"},{"messageHash":"0x085d659dca9c16e2bdf1cc5afe30a1e95e0c8d93bb369efa2c549c0768d224dc"},{"messageHash":"0x7b9057a2c40d82c3ddb5cfb1e3d048e42499f25821e8877932113b5201173c8e"},{"messageHash":"0xcf64628e32dde2f491e2280946b4832f8d7ed717f3d4133c3023b95d8f7ed8fa"},{"messageHash":"0x247b031b8d4335dc11fe4097013ef8fbdb0c165424f7718577fe949e535d5230"},{"messageHash":"0xc042a88041c6b8b64b897d76a079ed07c0b70bb1f61031910dafb4afceaaedf5"},{"messageHash":"0xb1059b8d6767ba76e62eda420b16f1516535a7877c959808d3942376b9b905a9"},{"messageHash":"0x397e5a51a5e9035f8c880ce213c86322957371a8dfc323d28b934f1fdb9cdca2"},{"messageHash":"0x6c3eace20331b16ea4f291c553b575e4cc1d4d3d7887733df62b78e88f00e40b"},{"messageHash":"0x3c42512001cfed57be1807495e3e24c5bd7f2f70da1a1a9f4e737f04e6277cad"}]}'
2026-03-31 02:27:18.329 DEBUG [src.steps.store] messages length is 10
2026-03-31 02:27:18.331 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-31 02:27:18.332 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-31 02:27:18.332 DEBUG [src.node.waku_node] Stopping container with id 1127a29c2663
2026-03-31 02:27:18.815 DEBUG [src.node.waku_node] Container stopped.
2026-03-31 02:27:18.815 DEBUG [src.node.waku_node] Stopping container with id dcee17e51abc
2026-03-31 02:27:19.278 DEBUG [src.node.waku_node] Container stopped.
2026-03-31 02:27:19.280 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-31 02:27:19.316 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-31 02:27:19.347 DEBUG [src.node.docker_mananger] No errors found in the waku logs.