183 lines
34 KiB
Plaintext

2026-03-03 04:35:31.507 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-03 04:35:31.508 DEBUG [tests.conftest] Running test: test_check_sync_when_2_nodes_publish with id: 2026-03-03_04-35-31__fbda2790-3a28-4200-a268-22cf25e5fbee
2026-03-03 04:35:31.508 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-03 04:35:31.518 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:35:31.518 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-03_04-35-31__fbda2790-3a28-4200-a268-22cf25e5fbee__wakuorg_nwaku:latest.log
2026-03-03 04:35:31.526 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:35:31.527 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-03_04-35-31__fbda2790-3a28-4200-a268-22cf25e5fbee__wakuorg_nwaku:latest.log
2026-03-03 04:35:31.536 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:35:31.537 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-03_04-35-31__fbda2790-3a28-4200-a268-22cf25e5fbee__wakuorg_nwaku:latest.log
2026-03-03 04:35:31.537 DEBUG [src.steps.store] Running fixture setup: store_setup
2026-03-03 04:35:31.538 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:35:31.538 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:35:31.539 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:35:31.540 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.140.144
2026-03-03 04:35:31.540 DEBUG [src.node.docker_mananger] Generated ports ['58947', '58948', '58949', '58950', '58951']
2026-03-03 04:35:31.540 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:35:31.540 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:35:31.540 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:35:31.540 DEBUG [src.node.docker_mananger] docker run -i -t -p 58947:58947 -p 58948:58948 -p 58949:58949 -p 58950:58950 -p 58951:58951 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=58949 --rest-port=58947 --tcp-port=58948 --discv5-udp-port=58950 --rest-address=0.0.0.0 --nat=extip:172.18.140.144 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=fc2a3b53cb902bfc6d3ac2fd1ccb84a5d727c2fab33f235980da9bedec8fc30e --store-sync=true --store=true --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=58951 --metrics-logging=true --relay=true
2026-03-03 04:35:31.728 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.140.144 waku 25a148b77c816ee4737a512239985e0d5f077fc5687c97a0777e0ac16559df6c
2026-03-03 04:35:31.742 ERROR [src.node.docker_mananger] Max retries reached for container 6c8741a72a04. Exiting log stream.
2026-03-03 04:35:31.768 DEBUG [src.node.docker_mananger] Container started with ID 25a148b77c81. Setting up logs at ./log/docker/node1_2026-03-03_04-35-31__fbda2790-3a28-4200-a268-22cf25e5fbee__wakuorg_nwaku:latest.log
2026-03-03 04:35:31.769 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 58947
2026-03-03 04:35:31.769 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:35:32.350 ERROR [src.node.docker_mananger] Max retries reached for container b2cb695732a6. Exiting log stream.
2026-03-03 04:35:32.771 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58947/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:35:32.774 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:35:32.774 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:35:32.774 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58947/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:35:32.776 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.140.144/tcp/58948/p2p/16Uiu2HAmDEYAiXETnSmygscW7z3mmwFcC4Jb3VJRboNvyAEzvWr5","/ip4/172.18.140.144/tcp/58949/ws/p2p/16Uiu2HAmDEYAiXETnSmygscW7z3mmwFcC4Jb3VJRboNvyAEzvWr5"],"enrUri":"enr:-L24QL7ntxSYL6I6NTRZrHypjAkwHmM2FMsJidzB-gXxWrvNYSyqowQ3L-E6r8IIk5PO5FcUkD9OESvCyZEHAfXXqiECgmlkgnY0gmlwhKwSjJCKbXVsdGlhZGRyc5YACASsEoyQBuZEAAoErBKMkAbmRd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDCJgkoawij98xdVrw8pGwVLb0z2uPnVHwawPgzUi1d2qDdGNwguZEg3VkcILmRoV3YWt1MhM"}'
2026-03-03 04:35:32.776 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:35:32.777 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:35:32.777 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:35:32.778 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:35:32.778 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.25.112
2026-03-03 04:35:32.778 DEBUG [src.node.docker_mananger] Generated ports ['39998', '39999', '40000', '40001', '40002']
2026-03-03 04:35:32.778 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:35:32.779 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:35:32.779 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:35:32.779 DEBUG [src.node.docker_mananger] docker run -i -t -p 39998:39998 -p 39999:39999 -p 40000:40000 -p 40001:40001 -p 40002:40002 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=40000 --rest-port=39998 --tcp-port=39999 --discv5-udp-port=40001 --rest-address=0.0.0.0 --nat=extip:172.18.25.112 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=3f2eaf6dbae7a0d7a8cade36bbcd599a24eccb5b1d7a82ec023fd9df247ecfc3 --store-sync=true --store=true --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=40002 --metrics-logging=true --relay=true --storenode=/ip4/172.18.140.144/tcp/58948/p2p/16Uiu2HAmDEYAiXETnSmygscW7z3mmwFcC4Jb3VJRboNvyAEzvWr5 --discv5-bootstrap-node=enr:-L24QL7ntxSYL6I6NTRZrHypjAkwHmM2FMsJidzB-gXxWrvNYSyqowQ3L-E6r8IIk5PO5FcUkD9OESvCyZEHAfXXqiECgmlkgnY0gmlwhKwSjJCKbXVsdGlhZGRyc5YACASsEoyQBuZEAAoErBKMkAbmRd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDCJgkoawij98xdVrw8pGwVLb0z2uPnVHwawPgzUi1d2qDdGNwguZEg3VkcILmRoV3YWt1MhM
2026-03-03 04:35:32.972 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.25.112 waku d8738897de4217c0d5dc8579ef35406d55a4612eca397bdf51b0babd844464ee
2026-03-03 04:35:33.006 DEBUG [src.node.docker_mananger] Container started with ID d8738897de42. Setting up logs at ./log/docker/node2_2026-03-03_04-35-31__fbda2790-3a28-4200-a268-22cf25e5fbee__wakuorg_nwaku:latest.log
2026-03-03 04:35:33.007 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 39998
2026-03-03 04:35:33.007 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:35:34.007 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39998/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:35:34.010 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:35:34.010 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:35:34.011 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39998/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:35:34.013 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.25.112/tcp/39999/p2p/16Uiu2HAmKAFes4aiBPrBM77jw5ZERCeEnobZTrCJdzKuh68bp6NK","/ip4/172.18.25.112/tcp/40000/ws/p2p/16Uiu2HAmKAFes4aiBPrBM77jw5ZERCeEnobZTrCJdzKuh68bp6NK"],"enrUri":"enr:-L24QLM9XTjzrB7S0xZ9o0j6fetB8Pl_49dCNcYe_2Y9fLjhGFjYUDqE_xNCg4uybfCmfASPj0cBdbUsyB7iHZhj6oYCgmlkgnY0gmlwhKwSGXCKbXVsdGlhZGRyc5YACASsEhlwBpw_AAoErBIZcAacQN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDYKVEp3GxFXrvwuRfThbMw8yGBcd2vfV9mYOBWAtWpqCDdGNwgpw_g3VkcIKcQYV3YWt1MhM"}'
2026-03-03 04:35:34.013 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:35:34.013 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:35:34.014 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:35:34.015 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:35:34.015 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.243.85
2026-03-03 04:35:34.015 DEBUG [src.node.docker_mananger] Generated ports ['52927', '52928', '52929', '52930', '52931']
2026-03-03 04:35:34.015 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:35:34.015 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:35:34.015 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:35:34.016 DEBUG [src.node.docker_mananger] docker run -i -t -p 52927:52927 -p 52928:52928 -p 52929:52929 -p 52930:52930 -p 52931:52931 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=52929 --rest-port=52927 --tcp-port=52928 --discv5-udp-port=52930 --rest-address=0.0.0.0 --nat=extip:172.18.243.85 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=bef8c6001926dc3bae97d22fef7fad0b3a4be6f83f5cc0fbf309c5bdf56fbdf5 --store-sync=true --store=false --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=52931 --metrics-logging=true --relay=false --storenode=/ip4/172.18.25.112/tcp/39999/p2p/16Uiu2HAmKAFes4aiBPrBM77jw5ZERCeEnobZTrCJdzKuh68bp6NK --discv5-bootstrap-node=enr:-L24QLM9XTjzrB7S0xZ9o0j6fetB8Pl_49dCNcYe_2Y9fLjhGFjYUDqE_xNCg4uybfCmfASPj0cBdbUsyB7iHZhj6oYCgmlkgnY0gmlwhKwSGXCKbXVsdGlhZGRyc5YACASsEhlwBpw_AAoErBIZcAacQN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDYKVEp3GxFXrvwuRfThbMw8yGBcd2vfV9mYOBWAtWpqCDdGNwgpw_g3VkcIKcQYV3YWt1MhM
2026-03-03 04:35:34.206 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.243.85 waku 5125d3450dce2a96454bdae82d6d587fc0bd40f315ad12a310fee7e803a7171b
2026-03-03 04:35:34.240 DEBUG [src.node.docker_mananger] Container started with ID 5125d3450dce. Setting up logs at ./log/docker/node3_2026-03-03_04-35-31__fbda2790-3a28-4200-a268-22cf25e5fbee__wakuorg_nwaku:latest.log
2026-03-03 04:35:34.241 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 52927
2026-03-03 04:35:34.241 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:35:35.241 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52927/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:35:35.244 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:35:35.244 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:35:35.244 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52927/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:35:35.247 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.243.85/tcp/52928/p2p/16Uiu2HAm7zJXSZ5RYBxZKFcr2jYNgq9cDkSkbnvkkZMhnhc19gRs","/ip4/172.18.243.85/tcp/52929/ws/p2p/16Uiu2HAm7zJXSZ5RYBxZKFcr2jYNgq9cDkSkbnvkkZMhnhc19gRs"],"enrUri":"enr:-L24QIoUors8WFopd0-A2WlZ7BhpnUdW7FhhbSZDvJUF80M2AGVX5y0tF43zB5Ld2mW1e04vWY75yosi-XPLj3DHFI4CgmlkgnY0gmlwhKwS81WKbXVsdGlhZGRyc5YACASsEvNVBs7AAAoErBLzVQbOwd0DgnJzhQADAQAAiXNlY3AyNTZrMaECuqgmvCD3Xsyb0DalvxpHUVShNTU2tSuEojPdGncyFr6DdGNwgs7Ag3VkcILOwoV3YWt1MgA"}'
2026-03-03 04:35:35.247 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:35:35.247 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39998/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.140.144/tcp/58948/p2p/16Uiu2HAmDEYAiXETnSmygscW7z3mmwFcC4Jb3VJRboNvyAEzvWr5"]'
2026-03-03 04:35:35.250 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.250 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:52927/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.25.112/tcp/39999/p2p/16Uiu2HAmKAFes4aiBPrBM77jw5ZERCeEnobZTrCJdzKuh68bp6NK"]'
2026-03-03 04:35:35.281 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.284 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58947/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-03 04:35:35.292 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.293 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39998/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-03 04:35:35.297 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.300 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.300 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58947/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.308 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.309 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.320 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.320 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58947/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.325 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.326 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.336 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.336 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58947/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.341 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.341 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.352 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.352 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58947/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.357 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.357 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.368 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.368 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58947/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.373 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.373 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.384 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.385 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58947/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.390 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.390 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.401 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.401 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58947/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.406 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.406 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.417 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.417 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58947/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.422 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.423 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.433 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.433 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58947/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.438 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.438 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.449 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.449 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:58947/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.455 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.456 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.467 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.467 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39998/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.473 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.473 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.484 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.484 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39998/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.490 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.491 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.501 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.501 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39998/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.506 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.507 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.518 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.518 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39998/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.523 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.523 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.534 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.534 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39998/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.541 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.541 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.551 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.552 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39998/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.557 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.558 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.569 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.569 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39998/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.574 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.574 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.585 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.585 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39998/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.590 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.591 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.601 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.601 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39998/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.607 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.607 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.618 DEBUG [src.steps.store] Relaying message
2026-03-03 04:35:35.618 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:39998/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-03 04:35:35.623 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:35:35.623 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-03 04:35:35.634 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-03 04:35:35.634 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:58947/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:35:35.637 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x565d7893bbcbb8ce289959815a1b821ee4e8c43409c2beb0ff5004b662b5bab4"},{"messageHash":"0xaa832b891cb4e2f09dfeccc5e0c79f412fb904c073a62c926af914da0a135362"},{"messageHash":"0x9507b5fa5de8c242de74418b1c2c2597c7f14215f1f216b11cf7a6eb04df4c8f"},{"messageHash":"0x4d501288ccf75ff9a28ee7029d8dd1aa8eee7eaef175d6640897c6336e3636bf"},{"messageHash":"0x927b006bcdf9ff0c6045b6280ecc45016c0dfa2a4ddcf2e854ee55e3cd0005a8"},{"messageHash":"0x3ca06005dccb1ccfa8f0fbda4d0dbabebcb9ed039633aee2d3adf9397ff92550"},{"messageHash":"0xfc91a30b04dc9b3eaf5dc53fa1078ab342f8517ca69372c0800d14302eee8e23"},{"messageHash":"0x48c0572821757a86abf92b3f1adba8aeca16fc095156777facca5879d583bf3f"},{"messageHash":"0x998496746c68ea3f260e2d78ca053ac8de011fc728d496e73ab268a4cfa0821f"},{"messageHash":"0x834a958af66fbd42bc79596df1ccb7a2ddf26b8445c1eb4cae0950f3d587cea4"},{"messageHash":"0x93435bc86b4e8b82fdea83ac0b444e6aab2b217706682f5647d1b415fe9ebfa3"},{"messageHash":"0xba0082bd001ad3e1def4298cd4c9b8523fd7db69f72683d60c73068ca512b1fc"},{"messageHash":"0xd7d505ad422452bcc58c43240e4d1f5de16584d7e58b7a6d41d24b8b4a71ac8b"},{"messageHash":"0x7e531328ede436abb127e5b263c5ed7ed57af34eb7adff2f708d01ac72730a31"},{"messageHash":"0x21329ee3d60af637ae697258144b6efd225cceaf36768756f6731d086ee92c6b"},{"messageHash":"0xb100b37a79fb3075bfe17f863f9e256edc92bbdaff31647c07422135e3cbe222"},{"messageHash":"0x86da85f07bb13ce53f0171c19ccb5182c08e931fc2ac524a2af72f0865e1658d"},{"messageHash":"0x1bd4ab91f2a2ed8baf6d9c820137e4e8743a743c64da85f1926355f607c88013"},{"messageHash":"0x7e91a0a9e445c471eba7c01c89830b64975542e91d5cc5441ce538e22ceb622b"},{"messageHash":"0x07d645f01257af055cca00a63711fba1b823b60d687c83947e6040cffb28768d"}]}'
2026-03-03 04:35:35.637 DEBUG [src.steps.store] messages length is 20
2026-03-03 04:35:35.639 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-03 04:35:35.639 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:39998/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:35:35.642 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x565d7893bbcbb8ce289959815a1b821ee4e8c43409c2beb0ff5004b662b5bab4"},{"messageHash":"0xaa832b891cb4e2f09dfeccc5e0c79f412fb904c073a62c926af914da0a135362"},{"messageHash":"0x9507b5fa5de8c242de74418b1c2c2597c7f14215f1f216b11cf7a6eb04df4c8f"},{"messageHash":"0x4d501288ccf75ff9a28ee7029d8dd1aa8eee7eaef175d6640897c6336e3636bf"},{"messageHash":"0x927b006bcdf9ff0c6045b6280ecc45016c0dfa2a4ddcf2e854ee55e3cd0005a8"},{"messageHash":"0x3ca06005dccb1ccfa8f0fbda4d0dbabebcb9ed039633aee2d3adf9397ff92550"},{"messageHash":"0xfc91a30b04dc9b3eaf5dc53fa1078ab342f8517ca69372c0800d14302eee8e23"},{"messageHash":"0x48c0572821757a86abf92b3f1adba8aeca16fc095156777facca5879d583bf3f"},{"messageHash":"0x998496746c68ea3f260e2d78ca053ac8de011fc728d496e73ab268a4cfa0821f"},{"messageHash":"0x834a958af66fbd42bc79596df1ccb7a2ddf26b8445c1eb4cae0950f3d587cea4"},{"messageHash":"0x93435bc86b4e8b82fdea83ac0b444e6aab2b217706682f5647d1b415fe9ebfa3"},{"messageHash":"0xba0082bd001ad3e1def4298cd4c9b8523fd7db69f72683d60c73068ca512b1fc"},{"messageHash":"0xd7d505ad422452bcc58c43240e4d1f5de16584d7e58b7a6d41d24b8b4a71ac8b"},{"messageHash":"0x7e531328ede436abb127e5b263c5ed7ed57af34eb7adff2f708d01ac72730a31"},{"messageHash":"0x21329ee3d60af637ae697258144b6efd225cceaf36768756f6731d086ee92c6b"},{"messageHash":"0xb100b37a79fb3075bfe17f863f9e256edc92bbdaff31647c07422135e3cbe222"},{"messageHash":"0x86da85f07bb13ce53f0171c19ccb5182c08e931fc2ac524a2af72f0865e1658d"},{"messageHash":"0x1bd4ab91f2a2ed8baf6d9c820137e4e8743a743c64da85f1926355f607c88013"},{"messageHash":"0x7e91a0a9e445c471eba7c01c89830b64975542e91d5cc5441ce538e22ceb622b"},{"messageHash":"0x07d645f01257af055cca00a63711fba1b823b60d687c83947e6040cffb28768d"}]}'
2026-03-03 04:35:35.642 DEBUG [src.steps.store] messages length is 20
2026-03-03 04:35:35.644 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-03 04:35:35.645 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52927/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:35:35.654 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"af6e5e227608ab35a7bf","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x565d7893bbcbb8ce289959815a1b821ee4e8c43409c2beb0ff5004b662b5bab4"},{"messageHash":"0xaa832b891cb4e2f09dfeccc5e0c79f412fb904c073a62c926af914da0a135362"},{"messageHash":"0x9507b5fa5de8c242de74418b1c2c2597c7f14215f1f216b11cf7a6eb04df4c8f"},{"messageHash":"0x4d501288ccf75ff9a28ee7029d8dd1aa8eee7eaef175d6640897c6336e3636bf"},{"messageHash":"0x927b006bcdf9ff0c6045b6280ecc45016c0dfa2a4ddcf2e854ee55e3cd0005a8"},{"messageHash":"0x3ca06005dccb1ccfa8f0fbda4d0dbabebcb9ed039633aee2d3adf9397ff92550"},{"messageHash":"0xfc91a30b04dc9b3eaf5dc53fa1078ab342f8517ca69372c0800d14302eee8e23"},{"messageHash":"0x48c0572821757a86abf92b3f1adba8aeca16fc095156777facca5879d583bf3f"},{"messageHash":"0x998496746c68ea3f260e2d78ca053ac8de011fc728d496e73ab268a4cfa0821f"},{"messageHash":"0x834a958af66fbd42bc79596df1ccb7a2ddf26b8445c1eb4cae0950f3d587cea4"},{"messageHash":"0x93435bc86b4e8b82fdea83ac0b444e6aab2b217706682f5647d1b415fe9ebfa3"},{"messageHash":"0xba0082bd001ad3e1def4298cd4c9b8523fd7db69f72683d60c73068ca512b1fc"},{"messageHash":"0xd7d505ad422452bcc58c43240e4d1f5de16584d7e58b7a6d41d24b8b4a71ac8b"},{"messageHash":"0x7e531328ede436abb127e5b263c5ed7ed57af34eb7adff2f708d01ac72730a31"},{"messageHash":"0x21329ee3d60af637ae697258144b6efd225cceaf36768756f6731d086ee92c6b"},{"messageHash":"0xb100b37a79fb3075bfe17f863f9e256edc92bbdaff31647c07422135e3cbe222"},{"messageHash":"0x86da85f07bb13ce53f0171c19ccb5182c08e931fc2ac524a2af72f0865e1658d"},{"messageHash":"0x1bd4ab91f2a2ed8baf6d9c820137e4e8743a743c64da85f1926355f607c88013"},{"messageHash":"0x7e91a0a9e445c471eba7c01c89830b64975542e91d5cc5441ce538e22ceb622b"},{"messageHash":"0x07d645f01257af055cca00a63711fba1b823b60d687c83947e6040cffb28768d"}]}'
2026-03-03 04:35:35.654 DEBUG [src.steps.store] messages length is 20
2026-03-03 04:35:35.657 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-03 04:35:35.659 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-03 04:35:35.659 DEBUG [src.node.waku_node] Stopping container with id 25a148b77c81
2026-03-03 04:35:36.224 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:35:36.225 DEBUG [src.node.waku_node] Stopping container with id d8738897de42
2026-03-03 04:35:36.790 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:35:36.792 DEBUG [src.node.waku_node] Stopping container with id 5125d3450dce
2026-03-03 04:35:37.079 ERROR [src.node.docker_mananger] Max retries reached for container 25a148b77c81. Exiting log stream.
2026-03-03 04:35:37.393 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:35:37.394 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-03 04:35:37.415 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-03 04:35:37.440 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-03 04:35:37.444 DEBUG [src.node.docker_mananger] No errors found in the waku logs.