183 lines
34 KiB
Plaintext

2026-03-20 04:36:11.795 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-20 04:36:11.795 DEBUG [tests.conftest] Running test: test_check_sync_when_2_nodes_publish with id: 2026-03-20_04-36-11__43a06925-8144-4bd2-9b22-6f35e840bedc
2026-03-20 04:36:11.795 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-20 04:36:11.802 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-20 04:36:11.802 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-20_04-36-11__43a06925-8144-4bd2-9b22-6f35e840bedc__wakuorg_nwaku:latest.log
2026-03-20 04:36:11.808 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-20 04:36:11.808 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-20_04-36-11__43a06925-8144-4bd2-9b22-6f35e840bedc__wakuorg_nwaku:latest.log
2026-03-20 04:36:11.814 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-20 04:36:11.814 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-20_04-36-11__43a06925-8144-4bd2-9b22-6f35e840bedc__wakuorg_nwaku:latest.log
2026-03-20 04:36:11.814 DEBUG [src.steps.store] Running fixture setup: store_setup
2026-03-20 04:36:11.815 DEBUG [src.node.waku_node] Starting Node...
2026-03-20 04:36:11.815 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-20 04:36:11.817 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-20 04:36:11.817 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.241.96
2026-03-20 04:36:11.817 DEBUG [src.node.docker_mananger] Generated ports ['7016', '7017', '7018', '7019', '7020']
2026-03-20 04:36:11.817 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-20 04:36:11.817 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-20 04:36:11.818 DEBUG [src.node.waku_node] Using volumes []
2026-03-20 04:36:11.818 DEBUG [src.node.docker_mananger] docker run -i -t -p 7016:7016 -p 7017:7017 -p 7018:7018 -p 7019:7019 -p 7020:7020 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=7018 --rest-port=7016 --tcp-port=7017 --discv5-udp-port=7019 --rest-address=0.0.0.0 --nat=extip:172.18.241.96 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=bfcc1b3af6cb9d89e29dad5fcb30456d582a4ca36d9ea428c18b86be80dca3bf --store-sync=true --store=true --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=7020 --metrics-logging=true --relay=true
2026-03-20 04:36:12.014 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.241.96 waku d1b2b31502988575fab82cf2b89f02baf8a09eb5d13f0a4db2e1498e267d61e4
2026-03-20 04:36:12.026 ERROR [src.node.docker_mananger] Max retries reached for container 52f06bfb59b4. Exiting log stream.
2026-03-20 04:36:12.053 DEBUG [src.node.docker_mananger] Container started with ID d1b2b3150298. Setting up logs at ./log/docker/node1_2026-03-20_04-36-11__43a06925-8144-4bd2-9b22-6f35e840bedc__wakuorg_nwaku:latest.log
2026-03-20 04:36:12.054 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 7016
2026-03-20 04:36:12.055 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-20 04:36:12.595 ERROR [src.node.docker_mananger] Max retries reached for container d4da35b77da3. Exiting log stream.
2026-03-20 04:36:13.056 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7016/health" -H "Content-Type: application/json" -d 'None'
2026-03-20 04:36:13.059 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-20 04:36:13.059 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-20 04:36:13.059 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7016/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-20 04:36:13.061 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.241.96/tcp/7017/p2p/16Uiu2HAm5s7GQHnsPTkbXPNdCuCmGnG6wTnT1mWnXCYahfQTKBQn","/ip4/172.18.241.96/tcp/7018/ws/p2p/16Uiu2HAm5s7GQHnsPTkbXPNdCuCmGnG6wTnT1mWnXCYahfQTKBQn"],"enrUri":"enr:-L24QET1f4-u2PUr4DQNYriT0MpDfIaPoGr0cK6jJQdEHz-nc8UavDfuHMkoa9alz5nH3lahoVYgdKiQB8ylvFZZRoQCgmlkgnY0gmlwhKwS8WCKbXVsdGlhZGRyc5YACASsEvFgBhtpAAoErBLxYAYbat0DgnJzhQADAQAAiXNlY3AyNTZrMaECmxj7Git66rjbUeNX6kkVp6kMiZ4HYqfyfytC5cnWCRuDdGNwghtpg3VkcIIba4V3YWt1MhM"}'
2026-03-20 04:36:13.062 INFO [src.node.waku_node] REST service is ready !!
2026-03-20 04:36:13.062 DEBUG [src.node.waku_node] Starting Node...
2026-03-20 04:36:13.062 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-20 04:36:13.063 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-20 04:36:13.064 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.65.119
2026-03-20 04:36:13.064 DEBUG [src.node.docker_mananger] Generated ports ['29271', '29272', '29273', '29274', '29275']
2026-03-20 04:36:13.064 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-20 04:36:13.064 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-20 04:36:13.064 DEBUG [src.node.waku_node] Using volumes []
2026-03-20 04:36:13.064 DEBUG [src.node.docker_mananger] docker run -i -t -p 29271:29271 -p 29272:29272 -p 29273:29273 -p 29274:29274 -p 29275:29275 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=29273 --rest-port=29271 --tcp-port=29272 --discv5-udp-port=29274 --rest-address=0.0.0.0 --nat=extip:172.18.65.119 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9f7edfb4a09de0abef6d10f1b256fc5cda6d7a82c7192c9bf3ae389afb7a5ce7 --store-sync=true --store=true --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=29275 --metrics-logging=true --relay=true --storenode=/ip4/172.18.241.96/tcp/7017/p2p/16Uiu2HAm5s7GQHnsPTkbXPNdCuCmGnG6wTnT1mWnXCYahfQTKBQn --discv5-bootstrap-node=enr:-L24QET1f4-u2PUr4DQNYriT0MpDfIaPoGr0cK6jJQdEHz-nc8UavDfuHMkoa9alz5nH3lahoVYgdKiQB8ylvFZZRoQCgmlkgnY0gmlwhKwS8WCKbXVsdGlhZGRyc5YACASsEvFgBhtpAAoErBLxYAYbat0DgnJzhQADAQAAiXNlY3AyNTZrMaECmxj7Git66rjbUeNX6kkVp6kMiZ4HYqfyfytC5cnWCRuDdGNwghtpg3VkcIIba4V3YWt1MhM
2026-03-20 04:36:13.259 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.65.119 waku 068b9047fbdf21b47e75975ed8f2e06f31c3d53d93d5a4d8ed0fd70043f6bdf5
2026-03-20 04:36:13.293 DEBUG [src.node.docker_mananger] Container started with ID 068b9047fbdf. Setting up logs at ./log/docker/node2_2026-03-20_04-36-11__43a06925-8144-4bd2-9b22-6f35e840bedc__wakuorg_nwaku:latest.log
2026-03-20 04:36:13.294 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 29271
2026-03-20 04:36:13.294 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-20 04:36:14.294 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29271/health" -H "Content-Type: application/json" -d 'None'
2026-03-20 04:36:14.297 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-20 04:36:14.298 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-20 04:36:14.298 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29271/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-20 04:36:14.300 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.65.119/tcp/29272/p2p/16Uiu2HAmNPqwP9Crrbuk2hgiZXsQxUog3hBo4LddtpAxJH6SwFAU","/ip4/172.18.65.119/tcp/29273/ws/p2p/16Uiu2HAmNPqwP9Crrbuk2hgiZXsQxUog3hBo4LddtpAxJH6SwFAU"],"enrUri":"enr:-L24QNrVx3ixizgSW-xpV5gcyZ3kCS4JJ7AjP7nEoLM8iENCasTKDVnaxnzhlnilNRjCRsdlrRXLlqxxDYIy8y_glowCgmlkgnY0gmlwhKwSQXeKbXVsdGlhZGRyc5YACASsEkF3BnJYAAoErBJBdwZyWd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDkLOlOsf97C5x-VT4UR5WFwxiatk4Ek_CPedIDfnKHL2DdGNwgnJYg3VkcIJyWoV3YWt1MhM"}'
2026-03-20 04:36:14.300 INFO [src.node.waku_node] REST service is ready !!
2026-03-20 04:36:14.300 DEBUG [src.node.waku_node] Starting Node...
2026-03-20 04:36:14.300 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-20 04:36:14.302 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-20 04:36:14.302 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.61.226
2026-03-20 04:36:14.302 DEBUG [src.node.docker_mananger] Generated ports ['9394', '9395', '9396', '9397', '9398']
2026-03-20 04:36:14.302 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-20 04:36:14.302 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-20 04:36:14.302 DEBUG [src.node.waku_node] Using volumes []
2026-03-20 04:36:14.303 DEBUG [src.node.docker_mananger] docker run -i -t -p 9394:9394 -p 9395:9395 -p 9396:9396 -p 9397:9397 -p 9398:9398 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=9396 --rest-port=9394 --tcp-port=9395 --discv5-udp-port=9397 --rest-address=0.0.0.0 --nat=extip:172.18.61.226 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=cafec1bbd3be0f8c198739e9453ad9aeeb7e2fb2ab7f6a7309d07e85eefb3ef0 --store-sync=true --store=false --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=9398 --metrics-logging=true --relay=false --storenode=/ip4/172.18.65.119/tcp/29272/p2p/16Uiu2HAmNPqwP9Crrbuk2hgiZXsQxUog3hBo4LddtpAxJH6SwFAU --discv5-bootstrap-node=enr:-L24QNrVx3ixizgSW-xpV5gcyZ3kCS4JJ7AjP7nEoLM8iENCasTKDVnaxnzhlnilNRjCRsdlrRXLlqxxDYIy8y_glowCgmlkgnY0gmlwhKwSQXeKbXVsdGlhZGRyc5YACASsEkF3BnJYAAoErBJBdwZyWd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDkLOlOsf97C5x-VT4UR5WFwxiatk4Ek_CPedIDfnKHL2DdGNwgnJYg3VkcIJyWoV3YWt1MhM
2026-03-20 04:36:14.500 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.61.226 waku 1600b17a5c70afd826e31a3e51874dec42c4ddd911cabe8689861a1f70e3a0c3
2026-03-20 04:36:14.537 DEBUG [src.node.docker_mananger] Container started with ID 1600b17a5c70. Setting up logs at ./log/docker/node3_2026-03-20_04-36-11__43a06925-8144-4bd2-9b22-6f35e840bedc__wakuorg_nwaku:latest.log
2026-03-20 04:36:14.538 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 9394
2026-03-20 04:36:14.539 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-20 04:36:15.539 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9394/health" -H "Content-Type: application/json" -d 'None'
2026-03-20 04:36:15.542 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-20 04:36:15.542 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-20 04:36:15.542 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9394/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-20 04:36:15.545 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.61.226/tcp/9395/p2p/16Uiu2HAmDMVQpamyh9sWEW9grfu4AMzsBYHQUNbiptbfr1Si96UM","/ip4/172.18.61.226/tcp/9396/ws/p2p/16Uiu2HAmDMVQpamyh9sWEW9grfu4AMzsBYHQUNbiptbfr1Si96UM"],"enrUri":"enr:-L24QPzYp1fyxJzqR0Ps8EWR4usRlK77FYJ-Z3PS-Km6ClG0HxQkBx0gDIwDi3Y90etKN-apD9A1ArapsoHt-JmlwmgCgmlkgnY0gmlwhKwSPeKKbXVsdGlhZGRyc5YACASsEj3iBiSzAAoErBI94gYktN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDCmAXc8Z0ebXgG9gvfbRh-eOqiDfQPfLbA5_8dw8tXdaDdGNwgiSzg3VkcIIktYV3YWt1MgA"}'
2026-03-20 04:36:15.545 INFO [src.node.waku_node] REST service is ready !!
2026-03-20 04:36:15.545 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29271/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.241.96/tcp/7017/p2p/16Uiu2HAm5s7GQHnsPTkbXPNdCuCmGnG6wTnT1mWnXCYahfQTKBQn"]'
2026-03-20 04:36:15.548 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.548 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:9394/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.65.119/tcp/29272/p2p/16Uiu2HAmNPqwP9Crrbuk2hgiZXsQxUog3hBo4LddtpAxJH6SwFAU"]'
2026-03-20 04:36:15.585 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.586 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7016/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-20 04:36:15.594 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.594 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29271/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-20 04:36:15.599 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.601 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.601 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7016/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.608 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.609 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.620 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.620 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7016/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.625 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.626 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.636 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.636 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7016/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.642 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.642 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.652 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.653 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7016/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.658 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.659 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.669 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.670 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7016/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.675 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.675 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.686 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.686 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7016/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.691 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.692 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.703 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.703 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7016/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.708 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.709 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.720 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.720 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7016/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.724 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.724 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.735 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.735 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7016/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.741 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.742 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.752 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.752 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7016/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.758 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.758 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.769 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.769 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29271/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.775 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.776 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.787 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.787 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29271/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.792 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.793 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.803 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.803 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29271/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.809 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.809 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.820 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.820 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29271/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.827 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.827 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.838 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.838 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29271/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.843 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.843 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.854 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.855 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29271/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.861 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.861 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.872 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.872 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29271/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.877 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.879 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.889 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.890 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29271/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.895 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.895 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.906 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.906 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29271/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.912 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.912 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.923 DEBUG [src.steps.store] Relaying message
2026-03-20 04:36:15.923 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29271/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-20 04:36:15.929 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-20 04:36:15.929 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-20 04:36:15.940 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-20 04:36:15.940 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7016/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-20 04:36:15.943 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x11a856258e57e3c36fc9a7d6ed4e9b284b34243dd3b42a9738c5ecf42645ad8d"},{"messageHash":"0xf6de9a0c225ecae1dfa419773fd12975abc02cfd19d445306ad9172e396444c0"},{"messageHash":"0x2a415bc32b23acd631c89744d1365a88047d7d141c3dee48f2c4174913c46a61"},{"messageHash":"0xccc7ef7893ddf560b1c9f5de9fb4410038c6239a796c4a9b2dd9ef1490bb90fb"},{"messageHash":"0xcd9ff30e0b424d1135efa1ebc295ce517b511b9f698b2cdc3cc8120c54b8ab78"},{"messageHash":"0x3e3ffd8e85b8894c37b33412b6c971d0fd0c42b256d13ee698319d8b029ce013"},{"messageHash":"0xcecac159d7d233942b69eec12647db010843145ccc44f00c4ed318ce255614b6"},{"messageHash":"0xeb0da0fef3e22cdf4143491e9538d304c1aed03ddc243397486f700292c716a7"},{"messageHash":"0x9777951fa98849dd7c60596068639a3ce45a0e629193e9328d68ea147c6b8a9b"},{"messageHash":"0x65e3d619c08ddc51be0701d9ee165bca5a764c8e7ea573f1a46ae7bde196a9ef"},{"messageHash":"0x21c6a6b5339f8523d6b5a590741f9ff6c101b8a8ecd4e2a79d86e6a232bf84b7"},{"messageHash":"0x13f3e679a667120f7c67dd162b4d7747e228e024afd27aee54468fac06ff3798"},{"messageHash":"0xe88f4e025887d6e1962b417a944b5e37eef70e048d17b0bfadf2daaaec2a4050"},{"messageHash":"0xd1ee139322f20e80375bb940b04cc4d1779786ff984b2214940bcfeef3764688"},{"messageHash":"0x4d673ebe481845002e84aa0b313cc9dc5a0129eccafc7dd566495689f8f727b9"},{"messageHash":"0x0ffb33b192b283899ae0e739debffee174b676f1827a67a4909ae51a1ad17577"},{"messageHash":"0x9febfff28ff089fcbeabdd8be8406d147576c08894d392bff19024b7672a7c79"},{"messageHash":"0xfbf2ea5c7d0ca5b6aee94419e2a78ddc88804828e4edfa06638969d1d0da7261"},{"messageHash":"0x6bff35325a316885c374ab4eeab4f9778d008d01b6ce10aa2bf72fd42ae1f925"},{"messageHash":"0xff02bd8b2906a33c66d4c6ad758d6bd5e493f630059e5a81faad58ccbb65f67c"}]}'
2026-03-20 04:36:15.943 DEBUG [src.steps.store] messages length is 20
2026-03-20 04:36:15.945 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-20 04:36:15.946 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29271/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-20 04:36:15.949 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x11a856258e57e3c36fc9a7d6ed4e9b284b34243dd3b42a9738c5ecf42645ad8d"},{"messageHash":"0xf6de9a0c225ecae1dfa419773fd12975abc02cfd19d445306ad9172e396444c0"},{"messageHash":"0x2a415bc32b23acd631c89744d1365a88047d7d141c3dee48f2c4174913c46a61"},{"messageHash":"0xccc7ef7893ddf560b1c9f5de9fb4410038c6239a796c4a9b2dd9ef1490bb90fb"},{"messageHash":"0xcd9ff30e0b424d1135efa1ebc295ce517b511b9f698b2cdc3cc8120c54b8ab78"},{"messageHash":"0x3e3ffd8e85b8894c37b33412b6c971d0fd0c42b256d13ee698319d8b029ce013"},{"messageHash":"0xcecac159d7d233942b69eec12647db010843145ccc44f00c4ed318ce255614b6"},{"messageHash":"0xeb0da0fef3e22cdf4143491e9538d304c1aed03ddc243397486f700292c716a7"},{"messageHash":"0x9777951fa98849dd7c60596068639a3ce45a0e629193e9328d68ea147c6b8a9b"},{"messageHash":"0x65e3d619c08ddc51be0701d9ee165bca5a764c8e7ea573f1a46ae7bde196a9ef"},{"messageHash":"0x21c6a6b5339f8523d6b5a590741f9ff6c101b8a8ecd4e2a79d86e6a232bf84b7"},{"messageHash":"0x13f3e679a667120f7c67dd162b4d7747e228e024afd27aee54468fac06ff3798"},{"messageHash":"0xe88f4e025887d6e1962b417a944b5e37eef70e048d17b0bfadf2daaaec2a4050"},{"messageHash":"0xd1ee139322f20e80375bb940b04cc4d1779786ff984b2214940bcfeef3764688"},{"messageHash":"0x4d673ebe481845002e84aa0b313cc9dc5a0129eccafc7dd566495689f8f727b9"},{"messageHash":"0x0ffb33b192b283899ae0e739debffee174b676f1827a67a4909ae51a1ad17577"},{"messageHash":"0x9febfff28ff089fcbeabdd8be8406d147576c08894d392bff19024b7672a7c79"},{"messageHash":"0xfbf2ea5c7d0ca5b6aee94419e2a78ddc88804828e4edfa06638969d1d0da7261"},{"messageHash":"0x6bff35325a316885c374ab4eeab4f9778d008d01b6ce10aa2bf72fd42ae1f925"},{"messageHash":"0xff02bd8b2906a33c66d4c6ad758d6bd5e493f630059e5a81faad58ccbb65f67c"}]}'
2026-03-20 04:36:15.949 DEBUG [src.steps.store] messages length is 20
2026-03-20 04:36:15.951 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-20 04:36:15.951 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9394/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-20 04:36:15.960 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"a18a17fcabfc9dabcaf3","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x11a856258e57e3c36fc9a7d6ed4e9b284b34243dd3b42a9738c5ecf42645ad8d"},{"messageHash":"0xf6de9a0c225ecae1dfa419773fd12975abc02cfd19d445306ad9172e396444c0"},{"messageHash":"0x2a415bc32b23acd631c89744d1365a88047d7d141c3dee48f2c4174913c46a61"},{"messageHash":"0xccc7ef7893ddf560b1c9f5de9fb4410038c6239a796c4a9b2dd9ef1490bb90fb"},{"messageHash":"0xcd9ff30e0b424d1135efa1ebc295ce517b511b9f698b2cdc3cc8120c54b8ab78"},{"messageHash":"0x3e3ffd8e85b8894c37b33412b6c971d0fd0c42b256d13ee698319d8b029ce013"},{"messageHash":"0xcecac159d7d233942b69eec12647db010843145ccc44f00c4ed318ce255614b6"},{"messageHash":"0xeb0da0fef3e22cdf4143491e9538d304c1aed03ddc243397486f700292c716a7"},{"messageHash":"0x9777951fa98849dd7c60596068639a3ce45a0e629193e9328d68ea147c6b8a9b"},{"messageHash":"0x65e3d619c08ddc51be0701d9ee165bca5a764c8e7ea573f1a46ae7bde196a9ef"},{"messageHash":"0x21c6a6b5339f8523d6b5a590741f9ff6c101b8a8ecd4e2a79d86e6a232bf84b7"},{"messageHash":"0x13f3e679a667120f7c67dd162b4d7747e228e024afd27aee54468fac06ff3798"},{"messageHash":"0xe88f4e025887d6e1962b417a944b5e37eef70e048d17b0bfadf2daaaec2a4050"},{"messageHash":"0xd1ee139322f20e80375bb940b04cc4d1779786ff984b2214940bcfeef3764688"},{"messageHash":"0x4d673ebe481845002e84aa0b313cc9dc5a0129eccafc7dd566495689f8f727b9"},{"messageHash":"0x0ffb33b192b283899ae0e739debffee174b676f1827a67a4909ae51a1ad17577"},{"messageHash":"0x9febfff28ff089fcbeabdd8be8406d147576c08894d392bff19024b7672a7c79"},{"messageHash":"0xfbf2ea5c7d0ca5b6aee94419e2a78ddc88804828e4edfa06638969d1d0da7261"},{"messageHash":"0x6bff35325a316885c374ab4eeab4f9778d008d01b6ce10aa2bf72fd42ae1f925"},{"messageHash":"0xff02bd8b2906a33c66d4c6ad758d6bd5e493f630059e5a81faad58ccbb65f67c"}]}'
2026-03-20 04:36:15.961 DEBUG [src.steps.store] messages length is 20
2026-03-20 04:36:15.964 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-20 04:36:15.966 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-20 04:36:15.966 DEBUG [src.node.waku_node] Stopping container with id d1b2b3150298
2026-03-20 04:36:16.554 DEBUG [src.node.waku_node] Container stopped.
2026-03-20 04:36:16.555 DEBUG [src.node.waku_node] Stopping container with id 068b9047fbdf
2026-03-20 04:36:17.110 DEBUG [src.node.waku_node] Container stopped.
2026-03-20 04:36:17.112 DEBUG [src.node.waku_node] Stopping container with id 1600b17a5c70
2026-03-20 04:36:17.366 ERROR [src.node.docker_mananger] Max retries reached for container d1b2b3150298. Exiting log stream.
2026-03-20 04:36:17.676 DEBUG [src.node.waku_node] Container stopped.
2026-03-20 04:36:17.679 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-20 04:36:17.689 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-20 04:36:17.701 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-20 04:36:17.705 DEBUG [src.node.docker_mananger] No errors found in the waku logs.