150 lines
27 KiB
Plaintext

2026-03-06 04:39:44.137 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-06 04:39:44.138 DEBUG [tests.conftest] Running test: test_sync_flags with id: 2026-03-06_04-39-44__b173fbd8-b5ec-4595-a6f7-acdb6a76b1b1
2026-03-06 04:39:44.138 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-06 04:39:44.146 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-06 04:39:44.146 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-06_04-39-44__b173fbd8-b5ec-4595-a6f7-acdb6a76b1b1__wakuorg_nwaku:latest.log
2026-03-06 04:39:44.152 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-06 04:39:44.152 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-06_04-39-44__b173fbd8-b5ec-4595-a6f7-acdb6a76b1b1__wakuorg_nwaku:latest.log
2026-03-06 04:39:44.159 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-06 04:39:44.159 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-06_04-39-44__b173fbd8-b5ec-4595-a6f7-acdb6a76b1b1__wakuorg_nwaku:latest.log
2026-03-06 04:39:44.159 DEBUG [src.steps.store] Running fixture setup: store_setup
2026-03-06 04:39:44.160 DEBUG [src.node.waku_node] Starting Node...
2026-03-06 04:39:44.160 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-06 04:39:44.162 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-06 04:39:44.162 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.68.237
2026-03-06 04:39:44.163 DEBUG [src.node.docker_mananger] Generated ports ['56230', '56231', '56232', '56233', '56234']
2026-03-06 04:39:44.163 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-06 04:39:44.163 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-06 04:39:44.164 DEBUG [src.node.waku_node] Using volumes []
2026-03-06 04:39:44.164 DEBUG [src.node.docker_mananger] docker run -i -t -p 56230:56230 -p 56231:56231 -p 56232:56232 -p 56233:56233 -p 56234:56234 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=56232 --rest-port=56230 --tcp-port=56231 --discv5-udp-port=56233 --rest-address=0.0.0.0 --nat=extip:172.18.68.237 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=5efedce4cac9a01dc228cff88a014eba2c2203c15e3ef7ab1895c8fe73237c6c --store-sync=true --store=true --store-sync-range=10 --store-sync-interval=1 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=56234 --metrics-logging=true --store-sync-relay-jitter=1 --relay=true
2026-03-06 04:39:44.367 ERROR [src.node.docker_mananger] Max retries reached for container 09fbf8c397a9. Exiting log stream.
2026-03-06 04:39:44.372 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.68.237 waku f8265f92a0f9de5813e6c2f21eaa166951d99c883da2bf43675d39f3e5c2765a
2026-03-06 04:39:44.407 DEBUG [src.node.docker_mananger] Container started with ID f8265f92a0f9. Setting up logs at ./log/docker/node1_2026-03-06_04-39-44__b173fbd8-b5ec-4595-a6f7-acdb6a76b1b1__wakuorg_nwaku:latest.log
2026-03-06 04:39:44.408 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 56230
2026-03-06 04:39:44.408 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-06 04:39:44.897 ERROR [src.node.docker_mananger] Max retries reached for container 87fa08d24c5b. Exiting log stream.
2026-03-06 04:39:45.408 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56230/health" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:39:45.411 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-06 04:39:45.411 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-06 04:39:45.412 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56230/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:39:45.414 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.68.237/tcp/56231/p2p/16Uiu2HAm4hFwHjm44ebszA7r21WdQAhRn6qBBoeHCkss32Bp7xxA","/ip4/172.18.68.237/tcp/56232/ws/p2p/16Uiu2HAm4hFwHjm44ebszA7r21WdQAhRn6qBBoeHCkss32Bp7xxA"],"enrUri":"enr:-L24QMTXNTPXAdIY7j4hN4l7nMRwBxrF_AsjA2OeB6H36MgNDMyVetgEL57zRYcjFsNAbI6CRfGsPcXqCIo-foCoM-UCgmlkgnY0gmlwhKwSRO2KbXVsdGlhZGRyc5YACASsEkTtBtunAAoErBJE7QbbqN0DgnJzhQADAQAAiXNlY3AyNTZrMaECibdKIctwntXiQr0uSIFG81fYaPCCYsX9AnqIDj8qP5uDdGNwgtung3VkcILbqYV3YWt1MhM"}'
2026-03-06 04:39:45.414 INFO [src.node.waku_node] REST service is ready !!
2026-03-06 04:39:45.414 DEBUG [src.node.waku_node] Starting Node...
2026-03-06 04:39:45.414 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-06 04:39:45.416 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-06 04:39:45.416 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.86.251
2026-03-06 04:39:45.416 DEBUG [src.node.docker_mananger] Generated ports ['6146', '6147', '6148', '6149', '6150']
2026-03-06 04:39:45.416 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-06 04:39:45.416 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-06 04:39:45.417 DEBUG [src.node.waku_node] Using volumes []
2026-03-06 04:39:45.417 DEBUG [src.node.docker_mananger] docker run -i -t -p 6146:6146 -p 6147:6147 -p 6148:6148 -p 6149:6149 -p 6150:6150 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=6148 --rest-port=6146 --tcp-port=6147 --discv5-udp-port=6149 --rest-address=0.0.0.0 --nat=extip:172.18.86.251 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=75aaebfe1ededbd87ffec94bffeadaf8e4f95e8fab4fc5caecbeafade89c3e17 --store-sync=true --store=false --store-sync-range=10 --store-sync-interval=1 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=6150 --metrics-logging=true --store-sync-relay-jitter=1 --relay=true --discv5-bootstrap-node=enr:-L24QMTXNTPXAdIY7j4hN4l7nMRwBxrF_AsjA2OeB6H36MgNDMyVetgEL57zRYcjFsNAbI6CRfGsPcXqCIo-foCoM-UCgmlkgnY0gmlwhKwSRO2KbXVsdGlhZGRyc5YACASsEkTtBtunAAoErBJE7QbbqN0DgnJzhQADAQAAiXNlY3AyNTZrMaECibdKIctwntXiQr0uSIFG81fYaPCCYsX9AnqIDj8qP5uDdGNwgtung3VkcILbqYV3YWt1MhM
2026-03-06 04:39:45.609 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.86.251 waku 08e412ed145cf06520bf6efec5f5002dfe81c655cd7020305c944b97f0177168
2026-03-06 04:39:45.641 DEBUG [src.node.docker_mananger] Container started with ID 08e412ed145c. Setting up logs at ./log/docker/node2_2026-03-06_04-39-44__b173fbd8-b5ec-4595-a6f7-acdb6a76b1b1__wakuorg_nwaku:latest.log
2026-03-06 04:39:45.641 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 6146
2026-03-06 04:39:45.641 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-06 04:39:46.642 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6146/health" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:39:46.671 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-06 04:39:46.672 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-06 04:39:46.672 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6146/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:39:46.678 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.86.251/tcp/6147/p2p/16Uiu2HAky2LW9XUbC61cqGvHHVpQ8FHgy4ZquAh47SJDvFK4KfGZ","/ip4/172.18.86.251/tcp/6148/ws/p2p/16Uiu2HAky2LW9XUbC61cqGvHHVpQ8FHgy4ZquAh47SJDvFK4KfGZ"],"enrUri":"enr:-L24QG2M44apnnJLNhImd8I-CLGMXcuLTJoag2kP-TdCMQ4GbIoH0PR_bled50P89BGK1M-tCLv5ntTsQdKVv2zibF8CgmlkgnY0gmlwhKwSVvuKbXVsdGlhZGRyc5YACASsElb7BhgDAAoErBJW-wYYBN0DgnJzhQADAQAAiXNlY3AyNTZrMaECNXRd5SvBbrA1LcUNFHGVjlbW8GXjUV_EqJe-m1G-yB6DdGNwghgDg3VkcIIYBYV3YWt1MgE"}'
2026-03-06 04:39:46.678 INFO [src.node.waku_node] REST service is ready !!
2026-03-06 04:39:46.679 DEBUG [src.node.waku_node] Starting Node...
2026-03-06 04:39:46.679 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-06 04:39:46.682 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-06 04:39:46.682 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.25.76
2026-03-06 04:39:46.682 DEBUG [src.node.docker_mananger] Generated ports ['12139', '12140', '12141', '12142', '12143']
2026-03-06 04:39:46.683 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-06 04:39:46.684 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-06 04:39:46.684 DEBUG [src.node.waku_node] Using volumes []
2026-03-06 04:39:46.684 DEBUG [src.node.docker_mananger] docker run -i -t -p 12139:12139 -p 12140:12140 -p 12141:12141 -p 12142:12142 -p 12143:12143 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=12141 --rest-port=12139 --tcp-port=12140 --discv5-udp-port=12142 --rest-address=0.0.0.0 --nat=extip:172.18.25.76 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=df1cdfaed7fdac8194c54e13af7efcaede13edeb2f42ddcffcf81ac3f2ab6a22 --store-sync=true --store=false --store-sync-range=10 --store-sync-interval=1 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=12143 --metrics-logging=true --store-sync-relay-jitter=1 --relay=true --discv5-bootstrap-node=enr:-L24QG2M44apnnJLNhImd8I-CLGMXcuLTJoag2kP-TdCMQ4GbIoH0PR_bled50P89BGK1M-tCLv5ntTsQdKVv2zibF8CgmlkgnY0gmlwhKwSVvuKbXVsdGlhZGRyc5YACASsElb7BhgDAAoErBJW-wYYBN0DgnJzhQADAQAAiXNlY3AyNTZrMaECNXRd5SvBbrA1LcUNFHGVjlbW8GXjUV_EqJe-m1G-yB6DdGNwghgDg3VkcIIYBYV3YWt1MgE
2026-03-06 04:39:46.887 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.25.76 waku d2f3b6b91f7e5e887ca2ea7eeffcf45fc4d7b50732c437ab3de77f3e22c1be78
2026-03-06 04:39:46.922 DEBUG [src.node.docker_mananger] Container started with ID d2f3b6b91f7e. Setting up logs at ./log/docker/node3_2026-03-06_04-39-44__b173fbd8-b5ec-4595-a6f7-acdb6a76b1b1__wakuorg_nwaku:latest.log
2026-03-06 04:39:46.922 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 12139
2026-03-06 04:39:46.923 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-06 04:39:47.925 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:12139/health" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:39:47.945 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-06 04:39:47.945 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-06 04:39:47.949 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:12139/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:39:47.955 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.25.76/tcp/12140/p2p/16Uiu2HAmBDFw3Azx9KiWGejmWgpet7gna7Rd6gmFeYERNMeFxxhn","/ip4/172.18.25.76/tcp/12141/ws/p2p/16Uiu2HAmBDFw3Azx9KiWGejmWgpet7gna7Rd6gmFeYERNMeFxxhn"],"enrUri":"enr:-L24QPYskdVJ4SC60Ugk9oq_j6SJDSJoXY4OLiz2ssE2CA-pXpRnO0PdZkT7OF6uMs-8j8G1_fqkABdmVbvTODIESb0CgmlkgnY0gmlwhKwSGUyKbXVsdGlhZGRyc5YACASsEhlMBi9sAAoErBIZTAYvbd0DgnJzhQADAQAAiXNlY3AyNTZrMaEC6ozUnIBHQsafM9lfokZ3IGbvMW3SuImotY5XmxWhWRGDdGNwgi9sg3VkcIIvboV3YWt1MgE"}'
2026-03-06 04:39:47.956 INFO [src.node.waku_node] REST service is ready !!
2026-03-06 04:39:47.956 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6146/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.68.237/tcp/56231/p2p/16Uiu2HAm4hFwHjm44ebszA7r21WdQAhRn6qBBoeHCkss32Bp7xxA"]'
2026-03-06 04:39:47.959 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:47.960 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12139/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.86.251/tcp/6147/p2p/16Uiu2HAky2LW9XUbC61cqGvHHVpQ8FHgy4ZquAh47SJDvFK4KfGZ"]'
2026-03-06 04:39:47.962 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:47.962 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56230/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-06 04:39:47.965 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:47.965 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6146/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-06 04:39:47.967 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:47.968 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:12139/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-06 04:39:47.970 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:47.970 DEBUG [src.steps.store] Relaying message
2026-03-06 04:39:47.971 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56230/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:39:47.976 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:47.977 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-06 04:39:48.179 DEBUG [src.steps.store] Relaying message
2026-03-06 04:39:48.179 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56230/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:39:48.185 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:48.186 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-06 04:39:48.387 DEBUG [src.steps.store] Relaying message
2026-03-06 04:39:48.387 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56230/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:39:48.393 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:48.393 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-06 04:39:48.595 DEBUG [src.steps.store] Relaying message
2026-03-06 04:39:48.595 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56230/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:39:48.601 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:48.601 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-06 04:39:48.802 DEBUG [src.steps.store] Relaying message
2026-03-06 04:39:48.802 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56230/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:39:48.808 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:48.810 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-06 04:39:49.011 DEBUG [src.steps.store] Relaying message
2026-03-06 04:39:49.011 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56230/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:39:49.018 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:49.019 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-06 04:39:49.220 DEBUG [src.steps.store] Relaying message
2026-03-06 04:39:49.220 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56230/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:39:49.226 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:49.227 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-06 04:39:49.428 DEBUG [src.steps.store] Relaying message
2026-03-06 04:39:49.429 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56230/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:39:49.435 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:49.437 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-06 04:39:49.637 DEBUG [src.steps.store] Relaying message
2026-03-06 04:39:49.638 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56230/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:39:49.644 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:49.644 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-06 04:39:49.845 DEBUG [src.steps.store] Relaying message
2026-03-06 04:39:49.845 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56230/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:39:49.851 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:39:49.852 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-06 04:39:50.054 DEBUG [src.libs.common] Sleeping for 2 seconds
2026-03-06 04:39:52.055 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-06 04:39:52.055 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56230/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:39:52.058 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0xdc04e042b2d837b3554c4e9815e1c860f420116d3fcda252f4d99861bda6abce"},{"messageHash":"0x5068bc94027014a16ab6552c2564b2165a1b2af77d440fe5f6a732c1761002ea"},{"messageHash":"0x959265fc1928a44405076e852d9a5971dc079bb0bb619a07d6f3b1270add52bc"},{"messageHash":"0x7b054bca1a0130aa1bf4ddfa9b38889df092b488f838379eeeb1ef597abd4c69"},{"messageHash":"0x9dc24136f2d560fc907aa909c0f7d4e819d9b63709392e25a1ea0671764f7412"},{"messageHash":"0xda73564ad637ca4a0493648599f8b36bec3a4a202ff85f44d6b93d7e0a900c34"},{"messageHash":"0x437afd84cda29d3dde352fb7134abbbd114245a9e0575457652539503ade09fc"},{"messageHash":"0x15917eab8722f9a393b22bfdebc34fb4383abc59e36c77cbbfa4b54dddddeca0"},{"messageHash":"0x5826e1a2bda5f0fe4c8b1fff36a5e92d4abd03c4333fd69316767708e4930a20"},{"messageHash":"0xe0865c3a07b686531b299d0d33b820c5ed98e435f302712648c9286998d61bbf"}]}'
2026-03-06 04:39:52.059 DEBUG [src.steps.store] messages length is 10
2026-03-06 04:39:52.060 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-06 04:39:52.060 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6146/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:39:52.069 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"c8d84350331f1b4cf0b9","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0xdc04e042b2d837b3554c4e9815e1c860f420116d3fcda252f4d99861bda6abce"},{"messageHash":"0x5068bc94027014a16ab6552c2564b2165a1b2af77d440fe5f6a732c1761002ea"},{"messageHash":"0x959265fc1928a44405076e852d9a5971dc079bb0bb619a07d6f3b1270add52bc"},{"messageHash":"0x7b054bca1a0130aa1bf4ddfa9b38889df092b488f838379eeeb1ef597abd4c69"},{"messageHash":"0x9dc24136f2d560fc907aa909c0f7d4e819d9b63709392e25a1ea0671764f7412"},{"messageHash":"0xda73564ad637ca4a0493648599f8b36bec3a4a202ff85f44d6b93d7e0a900c34"},{"messageHash":"0x437afd84cda29d3dde352fb7134abbbd114245a9e0575457652539503ade09fc"},{"messageHash":"0x15917eab8722f9a393b22bfdebc34fb4383abc59e36c77cbbfa4b54dddddeca0"},{"messageHash":"0x5826e1a2bda5f0fe4c8b1fff36a5e92d4abd03c4333fd69316767708e4930a20"},{"messageHash":"0xe0865c3a07b686531b299d0d33b820c5ed98e435f302712648c9286998d61bbf"}]}'
2026-03-06 04:39:52.070 DEBUG [src.steps.store] messages length is 10
2026-03-06 04:39:52.071 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-06 04:39:52.071 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:12139/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:39:52.075 ERROR [src.node.api_clients.base_client] HTTP error occurred: 412 Client Error: Precondition Failed for url: http://127.0.0.1:12139/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true. Response content: b'No suitable service peer & none discovered'
2026-03-06 04:39:52.126 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-06 04:39:52.128 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-06 04:39:52.128 DEBUG [src.node.waku_node] Stopping container with id f8265f92a0f9
2026-03-06 04:39:52.674 DEBUG [src.node.waku_node] Container stopped.
2026-03-06 04:39:52.675 DEBUG [src.node.waku_node] Stopping container with id 08e412ed145c
2026-03-06 04:39:53.273 DEBUG [src.node.waku_node] Container stopped.
2026-03-06 04:39:53.276 DEBUG [src.node.waku_node] Stopping container with id d2f3b6b91f7e
2026-03-06 04:39:53.493 ERROR [src.node.docker_mananger] Max retries reached for container f8265f92a0f9. Exiting log stream.
2026-03-06 04:39:53.833 DEBUG [src.node.waku_node] Container stopped.
2026-03-06 04:39:53.834 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-06 04:39:53.845 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-06 04:39:53.864 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-06 04:39:53.871 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-06 04:39:53.872 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-06 04:39:53.872 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-06 04:39:53.872 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-06_04-39-44__b173fbd8-b5ec-4595-a6f7-acdb6a76b1b1__wakuorg_nwaku:latest.log
2026-03-06 04:39:53.873 DEBUG [src.libs.common] Attaching file ./log/docker/node3_2026-03-06_04-39-44__b173fbd8-b5ec-4595-a6f7-acdb6a76b1b1__wakuorg_nwaku:latest.log
2026-03-06 04:39:53.874 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-06_04-39-44__b173fbd8-b5ec-4595-a6f7-acdb6a76b1b1__wakuorg_nwaku:latest.log