183 lines
34 KiB
Plaintext

2026-03-06 04:34:48.912 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-06 04:34:48.912 DEBUG [tests.conftest] Running test: test_check_sync_when_2_nodes_publish with id: 2026-03-06_04-34-48__398b4d9a-f723-4a9c-b132-d892cb2adaae
2026-03-06 04:34:48.913 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-06 04:34:48.920 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-06 04:34:48.920 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-06_04-34-48__398b4d9a-f723-4a9c-b132-d892cb2adaae__wakuorg_nwaku:latest.log
2026-03-06 04:34:48.926 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-06 04:34:48.927 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-06_04-34-48__398b4d9a-f723-4a9c-b132-d892cb2adaae__wakuorg_nwaku:latest.log
2026-03-06 04:34:48.933 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-06 04:34:48.933 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-06_04-34-48__398b4d9a-f723-4a9c-b132-d892cb2adaae__wakuorg_nwaku:latest.log
2026-03-06 04:34:48.933 DEBUG [src.steps.store] Running fixture setup: store_setup
2026-03-06 04:34:48.934 DEBUG [src.node.waku_node] Starting Node...
2026-03-06 04:34:48.934 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-06 04:34:48.936 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-06 04:34:48.937 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.195.124
2026-03-06 04:34:48.937 DEBUG [src.node.docker_mananger] Generated ports ['30366', '30367', '30368', '30369', '30370']
2026-03-06 04:34:48.937 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-06 04:34:48.937 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-06 04:34:48.937 DEBUG [src.node.waku_node] Using volumes []
2026-03-06 04:34:48.937 DEBUG [src.node.docker_mananger] docker run -i -t -p 30366:30366 -p 30367:30367 -p 30368:30368 -p 30369:30369 -p 30370:30370 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=30368 --rest-port=30366 --tcp-port=30367 --discv5-udp-port=30369 --rest-address=0.0.0.0 --nat=extip:172.18.195.124 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=4dba1ca2616c5ebf0df02bf471f7abe2ab06e73eeabddcfb834b6deadbf79abf --store-sync=true --store=true --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=30370 --metrics-logging=true --relay=true
2026-03-06 04:34:49.112 ERROR [src.node.docker_mananger] Max retries reached for container 2eb8f3be2c78. Exiting log stream.
2026-03-06 04:34:49.133 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.195.124 waku 5d0802e76c9b3175bbcf2994ceb1ef6317f913cbc60e687fe733ce78f63be915
2026-03-06 04:34:49.170 DEBUG [src.node.docker_mananger] Container started with ID 5d0802e76c9b. Setting up logs at ./log/docker/node1_2026-03-06_04-34-48__398b4d9a-f723-4a9c-b132-d892cb2adaae__wakuorg_nwaku:latest.log
2026-03-06 04:34:49.172 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 30366
2026-03-06 04:34:49.173 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-06 04:34:49.710 ERROR [src.node.docker_mananger] Max retries reached for container 8c390451d69b. Exiting log stream.
2026-03-06 04:34:50.175 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30366/health" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:34:50.178 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-06 04:34:50.179 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-06 04:34:50.179 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30366/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:34:50.182 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.195.124/tcp/30367/p2p/16Uiu2HAmCTKbvq5hWdE2nSprz2fjw8PRcuu8orR7V8RWBgbGwURA","/ip4/172.18.195.124/tcp/30368/ws/p2p/16Uiu2HAmCTKbvq5hWdE2nSprz2fjw8PRcuu8orR7V8RWBgbGwURA"],"enrUri":"enr:-L24QIvlztBK41wI8tFC_MbNl44UYKk94i0iyncf_-5PgQQrIreZwNIXe0uw4CktJ5rEHRzBcPblgu63nhMTma3ilmICgmlkgnY0gmlwhKwSw3yKbXVsdGlhZGRyc5YACASsEsN8BnafAAoErBLDfAZ2oN0DgnJzhQADAQAAiXNlY3AyNTZrMaEC_QLMjkPbQmivoq2F0aHBVRGTvOpRXKf5Z8Fv5FirFaWDdGNwgnafg3VkcIJ2oYV3YWt1MhM"}'
2026-03-06 04:34:50.182 INFO [src.node.waku_node] REST service is ready !!
2026-03-06 04:34:50.182 DEBUG [src.node.waku_node] Starting Node...
2026-03-06 04:34:50.182 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-06 04:34:50.184 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-06 04:34:50.184 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.102.85
2026-03-06 04:34:50.184 DEBUG [src.node.docker_mananger] Generated ports ['23360', '23361', '23362', '23363', '23364']
2026-03-06 04:34:50.184 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-06 04:34:50.184 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-06 04:34:50.185 DEBUG [src.node.waku_node] Using volumes []
2026-03-06 04:34:50.185 DEBUG [src.node.docker_mananger] docker run -i -t -p 23360:23360 -p 23361:23361 -p 23362:23362 -p 23363:23363 -p 23364:23364 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=23362 --rest-port=23360 --tcp-port=23361 --discv5-udp-port=23363 --rest-address=0.0.0.0 --nat=extip:172.18.102.85 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=45dcfd7eb7d4ff0a3fed811bc09569d01bb0b95b0dedda20de7afcc8a4eedea6 --store-sync=true --store=true --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=23364 --metrics-logging=true --relay=true --storenode=/ip4/172.18.195.124/tcp/30367/p2p/16Uiu2HAmCTKbvq5hWdE2nSprz2fjw8PRcuu8orR7V8RWBgbGwURA --discv5-bootstrap-node=enr:-L24QIvlztBK41wI8tFC_MbNl44UYKk94i0iyncf_-5PgQQrIreZwNIXe0uw4CktJ5rEHRzBcPblgu63nhMTma3ilmICgmlkgnY0gmlwhKwSw3yKbXVsdGlhZGRyc5YACASsEsN8BnafAAoErBLDfAZ2oN0DgnJzhQADAQAAiXNlY3AyNTZrMaEC_QLMjkPbQmivoq2F0aHBVRGTvOpRXKf5Z8Fv5FirFaWDdGNwgnafg3VkcIJ2oYV3YWt1MhM
2026-03-06 04:34:50.397 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.102.85 waku e09ec816059d451e41f5e5eaa809a023684907ecd55fa5aa686c68eee1161d43
2026-03-06 04:34:50.433 DEBUG [src.node.docker_mananger] Container started with ID e09ec816059d. Setting up logs at ./log/docker/node2_2026-03-06_04-34-48__398b4d9a-f723-4a9c-b132-d892cb2adaae__wakuorg_nwaku:latest.log
2026-03-06 04:34:50.433 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 23360
2026-03-06 04:34:50.434 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-06 04:34:51.434 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23360/health" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:34:51.437 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-06 04:34:51.438 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-06 04:34:51.438 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23360/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:34:51.440 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.102.85/tcp/23361/p2p/16Uiu2HAmKqTQTCBckhGuZ3Nas8n7PgFcgb6tdBBKvqQwEc2d7ZHh","/ip4/172.18.102.85/tcp/23362/ws/p2p/16Uiu2HAmKqTQTCBckhGuZ3Nas8n7PgFcgb6tdBBKvqQwEc2d7ZHh"],"enrUri":"enr:-L24QBGtFLxuWsUrbZONItRaP0l0LS_ZLM3KyUTKXpdoahxsSAt9hAESQvQCouQLslekkY-YuWlG38stgVVrgrSlLdkCgmlkgnY0gmlwhKwSZlWKbXVsdGlhZGRyc5YACASsEmZVBltBAAoErBJmVQZbQt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDarA1DMIYFLOf5AdKRyajwxmawfifG4WwfQmeL4JeyFiDdGNwgltBg3VkcIJbQ4V3YWt1MhM"}'
2026-03-06 04:34:51.440 INFO [src.node.waku_node] REST service is ready !!
2026-03-06 04:34:51.441 DEBUG [src.node.waku_node] Starting Node...
2026-03-06 04:34:51.441 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-06 04:34:51.442 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-06 04:34:51.442 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.142.31
2026-03-06 04:34:51.442 DEBUG [src.node.docker_mananger] Generated ports ['2700', '2701', '2702', '2703', '2704']
2026-03-06 04:34:51.443 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-06 04:34:51.443 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-06 04:34:51.443 DEBUG [src.node.waku_node] Using volumes []
2026-03-06 04:34:51.443 DEBUG [src.node.docker_mananger] docker run -i -t -p 2700:2700 -p 2701:2701 -p 2702:2702 -p 2703:2703 -p 2704:2704 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=2702 --rest-port=2700 --tcp-port=2701 --discv5-udp-port=2703 --rest-address=0.0.0.0 --nat=extip:172.18.142.31 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=46ba1efa9acddfacdafb529b7bb7bd505d63eda1bcecdfc9ea3edab172a9f787 --store-sync=true --store=false --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=2704 --metrics-logging=true --relay=false --storenode=/ip4/172.18.102.85/tcp/23361/p2p/16Uiu2HAmKqTQTCBckhGuZ3Nas8n7PgFcgb6tdBBKvqQwEc2d7ZHh --discv5-bootstrap-node=enr:-L24QBGtFLxuWsUrbZONItRaP0l0LS_ZLM3KyUTKXpdoahxsSAt9hAESQvQCouQLslekkY-YuWlG38stgVVrgrSlLdkCgmlkgnY0gmlwhKwSZlWKbXVsdGlhZGRyc5YACASsEmZVBltBAAoErBJmVQZbQt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDarA1DMIYFLOf5AdKRyajwxmawfifG4WwfQmeL4JeyFiDdGNwgltBg3VkcIJbQ4V3YWt1MhM
2026-03-06 04:34:51.650 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.142.31 waku f13a0b48bb5175e09486bf52918b5d75bbb0c302d3d8ebeed0afea62a5da8a77
2026-03-06 04:34:51.683 DEBUG [src.node.docker_mananger] Container started with ID f13a0b48bb51. Setting up logs at ./log/docker/node3_2026-03-06_04-34-48__398b4d9a-f723-4a9c-b132-d892cb2adaae__wakuorg_nwaku:latest.log
2026-03-06 04:34:51.684 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 2700
2026-03-06 04:34:51.684 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-06 04:34:52.684 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2700/health" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:34:52.687 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-06 04:34:52.688 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-06 04:34:52.688 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2700/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:34:52.690 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.142.31/tcp/2701/p2p/16Uiu2HAmHEWS4XaEkpwu8nLdr5bzBYfaftBfASiW1j3VQyZMwWnq","/ip4/172.18.142.31/tcp/2702/ws/p2p/16Uiu2HAmHEWS4XaEkpwu8nLdr5bzBYfaftBfASiW1j3VQyZMwWnq"],"enrUri":"enr:-L24QPmLJzVgnF6ABqdJWgEeqBYth7nuN6yZ5FYIYmcsLOmWD_AR0BgM9PhOJYCbn6W2-bpDv3OwY-NiLtmEma6tXLsCgmlkgnY0gmlwhKwSjh-KbXVsdGlhZGRyc5YACASsEo4fBgqNAAoErBKOHwYKjt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDRATttKeeFuH86v88MA8DfIQGEuohMM2xA2f7EF3V3KaDdGNwggqNg3VkcIIKj4V3YWt1MgA"}'
2026-03-06 04:34:52.690 INFO [src.node.waku_node] REST service is ready !!
2026-03-06 04:34:52.691 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23360/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.195.124/tcp/30367/p2p/16Uiu2HAmCTKbvq5hWdE2nSprz2fjw8PRcuu8orR7V8RWBgbGwURA"]'
2026-03-06 04:34:52.693 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.694 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:2700/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.102.85/tcp/23361/p2p/16Uiu2HAmKqTQTCBckhGuZ3Nas8n7PgFcgb6tdBBKvqQwEc2d7ZHh"]'
2026-03-06 04:34:52.729 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.734 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30366/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-06 04:34:52.746 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.748 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23360/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-06 04:34:52.752 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.753 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.753 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30366/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.760 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.760 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.771 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.772 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30366/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.778 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.778 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.788 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.789 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30366/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.794 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.794 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.805 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.805 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30366/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.811 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.811 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.822 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.822 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30366/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.828 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.828 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.839 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.839 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30366/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.845 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.846 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.856 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.857 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30366/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.862 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.863 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.874 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.874 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30366/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.880 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.881 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.891 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.892 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30366/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.897 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.898 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.909 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.909 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30366/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.914 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.915 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.925 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.926 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23360/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.931 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.932 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.942 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.942 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23360/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.948 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.948 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.959 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.959 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23360/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.965 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.965 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.976 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.976 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23360/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:52.983 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:52.983 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:52.994 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:52.994 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23360/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:53.000 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:53.001 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:53.012 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:53.012 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23360/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:53.018 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:53.018 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:53.029 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:53.029 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23360/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:53.034 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:53.035 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:53.045 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:53.046 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23360/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:53.051 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:53.051 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:53.063 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:53.063 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23360/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:53.069 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:53.070 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:53.080 DEBUG [src.steps.store] Relaying message
2026-03-06 04:34:53.081 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23360/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-06 04:34:53.086 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-06 04:34:53.087 DEBUG [src.libs.common] Sleeping for 0.01 seconds
2026-03-06 04:34:53.097 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-06 04:34:53.098 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30366/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:34:53.101 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x993470e396bf5c210e7bd0e878d6a81134e6b530c7f8933588d2b59cded5cf05"},{"messageHash":"0x75209283a4799d72d64a01b15f01b9ca239f2d24b73c26e348921f2150938eda"},{"messageHash":"0x93a93a00a66d154edb0e19c8de680d10bc640e79fe489e5c9cf307b06bbfbb7b"},{"messageHash":"0x7cc906a11ed9598c20d7cd29146fee3ee7dbe0f47fe245cd375bd990ef1b15ec"},{"messageHash":"0x8bffc581922ebb8375ca8920b300875c90f54e5aef05b953e95d04b3e6ebf9a2"},{"messageHash":"0x11a3a5914714051887a0f051527922e2389cfd7f54c8c347216eed0a60938d83"},{"messageHash":"0xde2c26c295d0bc1fdbae437fd82a146cca049f4502c90558b837b526495bc53d"},{"messageHash":"0xfa4b2241fffcf2a979683d8e964d6ec76b4686aad4ff708081e275dba63b4ffa"},{"messageHash":"0xffc54fa8a426e1408850108df47ca737d96b9a626a1551f378cfcc0bbbc71713"},{"messageHash":"0x0da48e3cee09488d0a58101e8bab3f3c9702eb838aa4acf99b2c5ba40ea4f2b5"},{"messageHash":"0xb3bb34df9a60caec70a0ecce0ddfdbb142e657174ea164e8ff4d0134999236ca"},{"messageHash":"0xb44002f91a1643db770072cf54687980d8f9bf927d8b219be83701342ae78c49"},{"messageHash":"0x9f0b6b4a9eb1be67ec01565dae1006a5833f93d89696fde68fe411e59dbf42a3"},{"messageHash":"0x34dee2c275072c53a73522f5873e4122ebf71685aab44d283d3380e51e430a55"},{"messageHash":"0x39aad68e40f4e3dc13d2b87b25022111b313861e0ef4a98554cba31b841fb966"},{"messageHash":"0xa28f458d9bfcc032fb3512d2dde2a379c0999194f398362dcbc4faaddf85956e"},{"messageHash":"0x4172dd8b4fadc3fb353929cff3633137d061cbbf6f4e27262669f3b29b847559"},{"messageHash":"0x03a9a43ddbbb8bec21e6bafce9107f0b6ff0065130b67f9f90a47b4b48f1dacc"},{"messageHash":"0x953b7e3cf51e5c052ac16b4e3baa6e85dfd8d6336f1b1da114ade054282a0387"},{"messageHash":"0x3c35ad498b292aff9576bfafcdeea7af250a86bb548372afdf6fb2f53d95dfa9"}]}'
2026-03-06 04:34:53.101 DEBUG [src.steps.store] messages length is 20
2026-03-06 04:34:53.103 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-06 04:34:53.104 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23360/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:34:53.107 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x993470e396bf5c210e7bd0e878d6a81134e6b530c7f8933588d2b59cded5cf05"},{"messageHash":"0x75209283a4799d72d64a01b15f01b9ca239f2d24b73c26e348921f2150938eda"},{"messageHash":"0x93a93a00a66d154edb0e19c8de680d10bc640e79fe489e5c9cf307b06bbfbb7b"},{"messageHash":"0x7cc906a11ed9598c20d7cd29146fee3ee7dbe0f47fe245cd375bd990ef1b15ec"},{"messageHash":"0x8bffc581922ebb8375ca8920b300875c90f54e5aef05b953e95d04b3e6ebf9a2"},{"messageHash":"0x11a3a5914714051887a0f051527922e2389cfd7f54c8c347216eed0a60938d83"},{"messageHash":"0xde2c26c295d0bc1fdbae437fd82a146cca049f4502c90558b837b526495bc53d"},{"messageHash":"0xfa4b2241fffcf2a979683d8e964d6ec76b4686aad4ff708081e275dba63b4ffa"},{"messageHash":"0xffc54fa8a426e1408850108df47ca737d96b9a626a1551f378cfcc0bbbc71713"},{"messageHash":"0x0da48e3cee09488d0a58101e8bab3f3c9702eb838aa4acf99b2c5ba40ea4f2b5"},{"messageHash":"0xb3bb34df9a60caec70a0ecce0ddfdbb142e657174ea164e8ff4d0134999236ca"},{"messageHash":"0xb44002f91a1643db770072cf54687980d8f9bf927d8b219be83701342ae78c49"},{"messageHash":"0x9f0b6b4a9eb1be67ec01565dae1006a5833f93d89696fde68fe411e59dbf42a3"},{"messageHash":"0x34dee2c275072c53a73522f5873e4122ebf71685aab44d283d3380e51e430a55"},{"messageHash":"0x39aad68e40f4e3dc13d2b87b25022111b313861e0ef4a98554cba31b841fb966"},{"messageHash":"0xa28f458d9bfcc032fb3512d2dde2a379c0999194f398362dcbc4faaddf85956e"},{"messageHash":"0x4172dd8b4fadc3fb353929cff3633137d061cbbf6f4e27262669f3b29b847559"},{"messageHash":"0x03a9a43ddbbb8bec21e6bafce9107f0b6ff0065130b67f9f90a47b4b48f1dacc"},{"messageHash":"0x953b7e3cf51e5c052ac16b4e3baa6e85dfd8d6336f1b1da114ade054282a0387"},{"messageHash":"0x3c35ad498b292aff9576bfafcdeea7af250a86bb548372afdf6fb2f53d95dfa9"}]}'
2026-03-06 04:34:53.107 DEBUG [src.steps.store] messages length is 20
2026-03-06 04:34:53.109 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-03-06 04:34:53.109 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2700/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=100&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-06 04:34:53.119 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"cb729ca444b1de538413","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x993470e396bf5c210e7bd0e878d6a81134e6b530c7f8933588d2b59cded5cf05"},{"messageHash":"0x75209283a4799d72d64a01b15f01b9ca239f2d24b73c26e348921f2150938eda"},{"messageHash":"0x93a93a00a66d154edb0e19c8de680d10bc640e79fe489e5c9cf307b06bbfbb7b"},{"messageHash":"0x7cc906a11ed9598c20d7cd29146fee3ee7dbe0f47fe245cd375bd990ef1b15ec"},{"messageHash":"0x8bffc581922ebb8375ca8920b300875c90f54e5aef05b953e95d04b3e6ebf9a2"},{"messageHash":"0x11a3a5914714051887a0f051527922e2389cfd7f54c8c347216eed0a60938d83"},{"messageHash":"0xde2c26c295d0bc1fdbae437fd82a146cca049f4502c90558b837b526495bc53d"},{"messageHash":"0xfa4b2241fffcf2a979683d8e964d6ec76b4686aad4ff708081e275dba63b4ffa"},{"messageHash":"0xffc54fa8a426e1408850108df47ca737d96b9a626a1551f378cfcc0bbbc71713"},{"messageHash":"0x0da48e3cee09488d0a58101e8bab3f3c9702eb838aa4acf99b2c5ba40ea4f2b5"},{"messageHash":"0xb3bb34df9a60caec70a0ecce0ddfdbb142e657174ea164e8ff4d0134999236ca"},{"messageHash":"0xb44002f91a1643db770072cf54687980d8f9bf927d8b219be83701342ae78c49"},{"messageHash":"0x9f0b6b4a9eb1be67ec01565dae1006a5833f93d89696fde68fe411e59dbf42a3"},{"messageHash":"0x34dee2c275072c53a73522f5873e4122ebf71685aab44d283d3380e51e430a55"},{"messageHash":"0x39aad68e40f4e3dc13d2b87b25022111b313861e0ef4a98554cba31b841fb966"},{"messageHash":"0xa28f458d9bfcc032fb3512d2dde2a379c0999194f398362dcbc4faaddf85956e"},{"messageHash":"0x4172dd8b4fadc3fb353929cff3633137d061cbbf6f4e27262669f3b29b847559"},{"messageHash":"0x03a9a43ddbbb8bec21e6bafce9107f0b6ff0065130b67f9f90a47b4b48f1dacc"},{"messageHash":"0x953b7e3cf51e5c052ac16b4e3baa6e85dfd8d6336f1b1da114ade054282a0387"},{"messageHash":"0x3c35ad498b292aff9576bfafcdeea7af250a86bb548372afdf6fb2f53d95dfa9"}]}'
2026-03-06 04:34:53.119 DEBUG [src.steps.store] messages length is 20
2026-03-06 04:34:53.122 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-06 04:34:53.124 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-06 04:34:53.124 DEBUG [src.node.waku_node] Stopping container with id 5d0802e76c9b
2026-03-06 04:34:53.684 DEBUG [src.node.waku_node] Container stopped.
2026-03-06 04:34:53.685 DEBUG [src.node.waku_node] Stopping container with id e09ec816059d
2026-03-06 04:34:54.304 DEBUG [src.node.waku_node] Container stopped.
2026-03-06 04:34:54.305 DEBUG [src.node.waku_node] Stopping container with id f13a0b48bb51
2026-03-06 04:34:54.542 ERROR [src.node.docker_mananger] Max retries reached for container 5d0802e76c9b. Exiting log stream.
2026-03-06 04:34:54.890 DEBUG [src.node.waku_node] Container stopped.
2026-03-06 04:34:54.892 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-06 04:34:54.915 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-06 04:34:54.929 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-06 04:34:54.934 DEBUG [src.node.docker_mananger] No errors found in the waku logs.