343 lines
59 KiB
Plaintext

2026-03-24 04:39:33.751 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-24 04:39:33.751 DEBUG [tests.conftest] Running test: test_multiple_pubsub_topics_same_cluster with id: 2026-03-24_04-39-33__156edbb3-98bc-4da2-b1a2-f7a409311d8b
2026-03-24 04:39:33.751 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-24 04:39:33.752 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-24 04:39:33.752 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-24 04:39:33.759 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-24 04:39:33.759 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-24_04-39-33__156edbb3-98bc-4da2-b1a2-f7a409311d8b__wakuorg_nwaku:latest.log
2026-03-24 04:39:33.759 DEBUG [src.node.waku_node] Starting Node...
2026-03-24 04:39:33.760 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-24 04:39:33.763 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-24 04:39:33.763 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.197.212
2026-03-24 04:39:33.763 DEBUG [src.node.docker_mananger] Generated ports ['25425', '25426', '25427', '25428', '25429']
2026-03-24 04:39:33.763 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-24 04:39:33.763 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-24 04:39:33.764 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-24 04:39:33.764 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-24 04:39:33.764 DEBUG [src.node.waku_node] Using volumes []
2026-03-24 04:39:33.764 DEBUG [src.node.docker_mananger] docker run -i -t -p 25425:25425 -p 25426:25426 -p 25427:25427 -p 25428:25428 -p 25429:25429 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=25427 --rest-port=25425 --tcp-port=25426 --discv5-udp-port=25428 --rest-address=0.0.0.0 --nat=extip:172.18.197.212 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=48249bac16eca4f6ed97448acf58009dd4e4ed99d83d690d85728455a04e0bde --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=25429 --metrics-logging=true --relay=true --filter=true
2026-03-24 04:39:33.955 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.197.212 waku 6dd76182ed1db830da741f33a8af8a690b04a027650a17338743ca029e4eb1c0
2026-03-24 04:39:33.993 DEBUG [src.node.docker_mananger] Container started with ID 6dd76182ed1d. Setting up logs at ./log/docker/node1_2026-03-24_04-39-33__156edbb3-98bc-4da2-b1a2-f7a409311d8b__wakuorg_nwaku:latest.log
2026-03-24 04:39:33.993 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 25425
2026-03-24 04:39:33.993 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-24 04:39:34.005 ERROR [src.node.docker_mananger] Max retries reached for container 27cf0c23201f. Exiting log stream.
2026-03-24 04:39:34.544 ERROR [src.node.docker_mananger] Max retries reached for container e6232718b339. Exiting log stream.
2026-03-24 04:39:34.995 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:25425/health" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:34.998 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-24 04:39:34.998 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-24 04:39:34.998 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:25425/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:35.001 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.197.212/tcp/25426/p2p/16Uiu2HAm8GZpXgXM1KGBp6hUnCT9stbmkjtLGZYrEarUh7TrSRh2","/ip4/172.18.197.212/tcp/25427/ws/p2p/16Uiu2HAm8GZpXgXM1KGBp6hUnCT9stbmkjtLGZYrEarUh7TrSRh2"],"enrUri":"enr:-L24QM_-wj-SZiTbsmnGi6cHCx9BKggkWRAyeJ-yR1OJGxE3fnivQXHZpqC9FjAV0ar6Q3ySuVkFm3ta6kmi0lruZk4CgmlkgnY0gmlwhKwSxdSKbXVsdGlhZGRyc5YACASsEsXUBmNSAAoErBLF1AZjU90DgnJzhQACAQAAiXNlY3AyNTZrMaECvtK86epS1V_KPl0tJSGZhtawhAn40xoUqDuTeUZiPymDdGNwgmNSg3VkcIJjVIV3YWt1MgU"}'
2026-03-24 04:39:35.002 INFO [src.node.waku_node] REST service is ready !!
2026-03-24 04:39:35.008 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-24 04:39:35.009 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-24_04-39-33__156edbb3-98bc-4da2-b1a2-f7a409311d8b__wakuorg_nwaku:latest.log
2026-03-24 04:39:35.009 DEBUG [src.node.waku_node] Starting Node...
2026-03-24 04:39:35.009 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-24 04:39:35.010 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-24 04:39:35.010 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.255.59
2026-03-24 04:39:35.010 DEBUG [src.node.docker_mananger] Generated ports ['36106', '36107', '36108', '36109', '36110']
2026-03-24 04:39:35.011 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-24 04:39:35.011 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-24 04:39:35.011 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-24 04:39:35.011 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-24 04:39:35.011 DEBUG [src.node.waku_node] Using volumes []
2026-03-24 04:39:35.011 DEBUG [src.node.docker_mananger] docker run -i -t -p 36106:36106 -p 36107:36107 -p 36108:36108 -p 36109:36109 -p 36110:36110 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=36108 --rest-port=36106 --tcp-port=36107 --discv5-udp-port=36109 --rest-address=0.0.0.0 --nat=extip:172.18.255.59 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=e8ad0aaebe8e6f7dfe53d88aea7c4baae981fbfd4b8f6f5c62a8e7cd6cda5f94 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=36110 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QM_-wj-SZiTbsmnGi6cHCx9BKggkWRAyeJ-yR1OJGxE3fnivQXHZpqC9FjAV0ar6Q3ySuVkFm3ta6kmi0lruZk4CgmlkgnY0gmlwhKwSxdSKbXVsdGlhZGRyc5YACASsEsXUBmNSAAoErBLF1AZjU90DgnJzhQACAQAAiXNlY3AyNTZrMaECvtK86epS1V_KPl0tJSGZhtawhAn40xoUqDuTeUZiPymDdGNwgmNSg3VkcIJjVIV3YWt1MgU
2026-03-24 04:39:35.211 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.255.59 waku de0be96e74b371873128333c27e4c3915890754447ff01e62380f1fc90839102
2026-03-24 04:39:35.244 DEBUG [src.node.docker_mananger] Container started with ID de0be96e74b3. Setting up logs at ./log/docker/node2_2026-03-24_04-39-33__156edbb3-98bc-4da2-b1a2-f7a409311d8b__wakuorg_nwaku:latest.log
2026-03-24 04:39:35.245 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 36106
2026-03-24 04:39:35.245 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-24 04:39:36.245 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36106/health" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:36.248 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-24 04:39:36.248 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-24 04:39:36.249 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36106/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:36.251 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.255.59/tcp/36107/p2p/16Uiu2HAmRCuzSsbAZYgDppbEcvdabvKepHxP6Cgnomj4qy3kecAr","/ip4/172.18.255.59/tcp/36108/ws/p2p/16Uiu2HAmRCuzSsbAZYgDppbEcvdabvKepHxP6Cgnomj4qy3kecAr"],"enrUri":"enr:-L24QOMKT3cvOPXZzN-JRg7etzjd6y9V5aZPn743ufv29ylbTLiHQ2sDxR-WIttD3xPgKg5Adj9FEf9bsSoKUEu4cugCgmlkgnY0gmlwhKwS_zuKbXVsdGlhZGRyc5YACASsEv87Bo0LAAoErBL_OwaNDN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDunnmXUGq1VPYkjmm8efWKu3E3OFbGHulwjw5VgIzf9-DdGNwgo0Lg3VkcIKNDYV3YWt1MgE"}'
2026-03-24 04:39:36.251 INFO [src.node.waku_node] REST service is ready !!
2026-03-24 04:39:36.252 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:36106/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.197.212/tcp/25426/p2p/16Uiu2HAm8GZpXgXM1KGBp6hUnCT9stbmkjtLGZYrEarUh7TrSRh2"]'
2026-03-24 04:39:36.273 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:36.275 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:25425/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-24 04:39:36.290 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:36.292 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:36106/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-24 04:39:36.296 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:36.297 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:25425/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:36.308 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:36.309 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:36.409 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:36.410 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:25425/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:36.413 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327176297228434,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:36.415 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:36.416 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36106/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:36.419 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327176297228434,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:36.421 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:25425/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:36.428 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:36.429 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:36.529 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:36.530 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:25425/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:36.533 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327176420951374,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:36.534 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:36.534 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36106/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:36.537 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327176420951374,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:36.538 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:25425/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:36.546 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:36.546 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:36.648 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:36.648 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:25425/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:36.651 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327176538734244,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:36.653 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:36.653 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36106/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:36.655 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327176538734244,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:36.657 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:25425/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F3" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:36.665 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:36.665 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:36.765 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:36.766 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:25425/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F3" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:36.768 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327176657219272,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:36.770 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:36.771 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36106/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F3" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:36.773 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327176657219272,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:36.774 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:25425/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F4" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:36.782 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:36.783 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:36.883 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:36.883 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:25425/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F4" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:36.886 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/statusim/1/community/cbor","version":0,"timestamp":1774327173133053049,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327176774824328,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:36.894 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-24 04:39:36.895 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-24 04:39:36.895 DEBUG [src.node.waku_node] Stopping container with id 6dd76182ed1d
2026-03-24 04:39:37.486 DEBUG [src.node.waku_node] Container stopped.
2026-03-24 04:39:37.486 DEBUG [src.node.waku_node] Stopping container with id de0be96e74b3
2026-03-24 04:39:38.069 DEBUG [src.node.waku_node] Container stopped.
2026-03-24 04:39:38.070 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-24 04:39:38.099 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-24 04:39:38.132 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-24 04:39:38.134 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-24 04:39:38.134 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-24 04:39:38.135 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-24_04-39-33__156edbb3-98bc-4da2-b1a2-f7a409311d8b__wakuorg_nwaku:latest.log
2026-03-24 04:39:38.136 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-24_04-39-33__156edbb3-98bc-4da2-b1a2-f7a409311d8b__wakuorg_nwaku:latest.log
2026-03-24 04:39:38.141 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-24 04:39:38.141 DEBUG [tests.conftest] Running test: test_multiple_pubsub_topics_same_cluster with id: 2026-03-24_04-39-38__c759f07e-f5c6-4d12-bdfa-ed095b25de85
2026-03-24 04:39:38.141 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-24 04:39:38.142 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-24 04:39:38.142 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-24 04:39:38.149 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-24 04:39:38.149 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-24_04-39-38__c759f07e-f5c6-4d12-bdfa-ed095b25de85__wakuorg_nwaku:latest.log
2026-03-24 04:39:38.150 DEBUG [src.node.waku_node] Starting Node...
2026-03-24 04:39:38.150 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-24 04:39:38.151 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-24 04:39:38.152 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.88.80
2026-03-24 04:39:38.152 DEBUG [src.node.docker_mananger] Generated ports ['52459', '52460', '52461', '52462', '52463']
2026-03-24 04:39:38.152 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-24 04:39:38.152 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-24 04:39:38.152 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-24 04:39:38.152 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-24 04:39:38.153 DEBUG [src.node.waku_node] Using volumes []
2026-03-24 04:39:38.153 DEBUG [src.node.docker_mananger] docker run -i -t -p 52459:52459 -p 52460:52460 -p 52461:52461 -p 52462:52462 -p 52463:52463 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=52461 --rest-port=52459 --tcp-port=52460 --discv5-udp-port=52462 --rest-address=0.0.0.0 --nat=extip:172.18.88.80 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=97966bed1ce27ac5b55ac22bb9be1ae1ffc4db4225fc0fffb5f7a8bbab0ad93b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=52463 --metrics-logging=true --relay=true --filter=true
2026-03-24 04:39:38.336 ERROR [src.node.docker_mananger] Max retries reached for container 6dd76182ed1d. Exiting log stream.
2026-03-24 04:39:38.350 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.88.80 waku 23d52bf2c38345c43c190f74f8b4488c5290760ac8bcb5bd29a79e9a0887c35f
2026-03-24 04:39:38.386 DEBUG [src.node.docker_mananger] Container started with ID 23d52bf2c383. Setting up logs at ./log/docker/node1_2026-03-24_04-39-38__c759f07e-f5c6-4d12-bdfa-ed095b25de85__wakuorg_nwaku:latest.log
2026-03-24 04:39:38.387 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 52459
2026-03-24 04:39:38.387 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-24 04:39:38.911 ERROR [src.node.docker_mananger] Max retries reached for container de0be96e74b3. Exiting log stream.
2026-03-24 04:39:39.387 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52459/health" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:39.390 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-24 04:39:39.390 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-24 04:39:39.391 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52459/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:39.393 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.88.80/tcp/52460/p2p/16Uiu2HAmNnKnh8jpdHpPt93NKZt1wKPbfwNbgdvKCQwNE7r3eMpN","/ip4/172.18.88.80/tcp/52461/ws/p2p/16Uiu2HAmNnKnh8jpdHpPt93NKZt1wKPbfwNbgdvKCQwNE7r3eMpN"],"enrUri":"enr:-L24QB9crVyMrHmynoV326E-nGGl72BcGUEmoOI0L_7WaEGMEet4uVDFstCvQJu_IxnkVz0woucPt0lgksvwdauDcbkCgmlkgnY0gmlwhKwSWFCKbXVsdGlhZGRyc5YACASsElhQBszsAAoErBJYUAbM7d0DgnJzhQACAQAAiXNlY3AyNTZrMaEDlnXp3JcminWLU04GjfEWIsaWS-1glrVfHMd966XCihODdGNwgszsg3VkcILM7oV3YWt1MgU"}'
2026-03-24 04:39:39.393 INFO [src.node.waku_node] REST service is ready !!
2026-03-24 04:39:39.400 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-24 04:39:39.401 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-24_04-39-38__c759f07e-f5c6-4d12-bdfa-ed095b25de85__wakuorg_nwaku:latest.log
2026-03-24 04:39:39.401 DEBUG [src.node.waku_node] Starting Node...
2026-03-24 04:39:39.401 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-24 04:39:39.402 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-24 04:39:39.402 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.125.79
2026-03-24 04:39:39.402 DEBUG [src.node.docker_mananger] Generated ports ['43732', '43733', '43734', '43735', '43736']
2026-03-24 04:39:39.403 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-24 04:39:39.403 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-24 04:39:39.403 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-24 04:39:39.403 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-24 04:39:39.403 DEBUG [src.node.waku_node] Using volumes []
2026-03-24 04:39:39.403 DEBUG [src.node.docker_mananger] docker run -i -t -p 43732:43732 -p 43733:43733 -p 43734:43734 -p 43735:43735 -p 43736:43736 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=43734 --rest-port=43732 --tcp-port=43733 --discv5-udp-port=43735 --rest-address=0.0.0.0 --nat=extip:172.18.125.79 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=4a14905c6de4dd2fa6f7f624f648bd2db4d20fcddaede4bfd91b4b14ea6e25d1 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=43736 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QB9crVyMrHmynoV326E-nGGl72BcGUEmoOI0L_7WaEGMEet4uVDFstCvQJu_IxnkVz0woucPt0lgksvwdauDcbkCgmlkgnY0gmlwhKwSWFCKbXVsdGlhZGRyc5YACASsElhQBszsAAoErBJYUAbM7d0DgnJzhQACAQAAiXNlY3AyNTZrMaEDlnXp3JcminWLU04GjfEWIsaWS-1glrVfHMd966XCihODdGNwgszsg3VkcILM7oV3YWt1MgU
2026-03-24 04:39:39.619 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.125.79 waku 41228d447eb5c8dda43238e5ea1e4b014623bdcdcd53991e650c205abcc19f09
2026-03-24 04:39:39.658 DEBUG [src.node.docker_mananger] Container started with ID 41228d447eb5. Setting up logs at ./log/docker/node2_2026-03-24_04-39-38__c759f07e-f5c6-4d12-bdfa-ed095b25de85__wakuorg_nwaku:latest.log
2026-03-24 04:39:39.658 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 43732
2026-03-24 04:39:39.658 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-24 04:39:40.659 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43732/health" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:40.661 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-24 04:39:40.762 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43732/health" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:40.764 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-24 04:39:40.865 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43732/health" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:40.868 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-24 04:39:40.969 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43732/health" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:40.971 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-24 04:39:41.072 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43732/health" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:41.087 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-24 04:39:41.187 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43732/health" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:41.190 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-24 04:39:41.190 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-24 04:39:41.190 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:43732/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:41.192 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.125.79/tcp/43733/p2p/16Uiu2HAmLCgytgu41efdnXR8ma6LkyscQbL2ypbe6H42YtpELyhL","/ip4/172.18.125.79/tcp/43734/ws/p2p/16Uiu2HAmLCgytgu41efdnXR8ma6LkyscQbL2ypbe6H42YtpELyhL"],"enrUri":"enr:-L24QIFyuFGBrpv5cPNwXPSg_z3NFqOXTe_O9F5fT-h2r8ZTCrN2iTvDrS1YJoAMCEo6zwgFGnK41psl6Zrsft7OhbMCgmlkgnY0gmlwhKwSfU-KbXVsdGlhZGRyc5YACASsEn1PBqrVAAoErBJ9Twaq1t0DgnJzhQACAQAAiXNlY3AyNTZrMaEDcCDBWPzlgXD4-CAj2ayelmlJWaoKIWxLaz9FnlwQx2uDdGNwgqrVg3VkcIKq14V3YWt1MgE"}'
2026-03-24 04:39:41.193 INFO [src.node.waku_node] REST service is ready !!
2026-03-24 04:39:41.193 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:43732/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.88.80/tcp/52460/p2p/16Uiu2HAmNnKnh8jpdHpPt93NKZt1wKPbfwNbgdvKCQwNE7r3eMpN"]'
2026-03-24 04:39:41.216 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:41.218 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:52459/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-24 04:39:41.226 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:41.228 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:43732/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-24 04:39:41.235 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:41.236 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:52459/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:41.246 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:41.247 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:41.347 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:41.348 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52459/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:41.350 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"L3dha3UvMi9ycy8yLzA=","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327177424268622,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327176297228434,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327181236132357,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:41.358 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-24 04:39:41.359 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-24 04:39:41.359 DEBUG [src.node.waku_node] Stopping container with id 23d52bf2c383
2026-03-24 04:39:41.962 DEBUG [src.node.waku_node] Container stopped.
2026-03-24 04:39:41.963 DEBUG [src.node.waku_node] Stopping container with id 41228d447eb5
2026-03-24 04:39:42.527 DEBUG [src.node.waku_node] Container stopped.
2026-03-24 04:39:42.529 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-24 04:39:42.568 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-24 04:39:42.597 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-24 04:39:42.598 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-24 04:39:42.599 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-24 04:39:42.600 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-24_04-39-38__c759f07e-f5c6-4d12-bdfa-ed095b25de85__wakuorg_nwaku:latest.log
2026-03-24 04:39:42.601 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-24_04-39-38__c759f07e-f5c6-4d12-bdfa-ed095b25de85__wakuorg_nwaku:latest.log
2026-03-24 04:39:42.605 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-24 04:39:42.605 DEBUG [tests.conftest] Running test: test_multiple_pubsub_topics_same_cluster with id: 2026-03-24_04-39-42__958d3707-b578-42b6-8e02-7ce9eec4bb6a
2026-03-24 04:39:42.605 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-24 04:39:42.606 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-24 04:39:42.606 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-24 04:39:42.613 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-24 04:39:42.613 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-24_04-39-42__958d3707-b578-42b6-8e02-7ce9eec4bb6a__wakuorg_nwaku:latest.log
2026-03-24 04:39:42.613 DEBUG [src.node.waku_node] Starting Node...
2026-03-24 04:39:42.613 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-24 04:39:42.615 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-24 04:39:42.615 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.179.228
2026-03-24 04:39:42.615 DEBUG [src.node.docker_mananger] Generated ports ['13621', '13622', '13623', '13624', '13625']
2026-03-24 04:39:42.615 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-24 04:39:42.615 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-24 04:39:42.616 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-24 04:39:42.616 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-24 04:39:42.616 DEBUG [src.node.waku_node] Using volumes []
2026-03-24 04:39:42.616 DEBUG [src.node.docker_mananger] docker run -i -t -p 13621:13621 -p 13622:13622 -p 13623:13623 -p 13624:13624 -p 13625:13625 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=13623 --rest-port=13621 --tcp-port=13622 --discv5-udp-port=13624 --rest-address=0.0.0.0 --nat=extip:172.18.179.228 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=ebea5ec214eaffdea982cdcc83cdd0cef0fe4e0bfc92e4920dded2c32ccfd26b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=13625 --metrics-logging=true --relay=true --filter=true
2026-03-24 04:39:42.802 ERROR [src.node.docker_mananger] Max retries reached for container 23d52bf2c383. Exiting log stream.
2026-03-24 04:39:42.819 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.179.228 waku 5bddd2bfe437abe9c6a6559bed8c62420c413270247e73f364cf389d69d5af51
2026-03-24 04:39:42.857 DEBUG [src.node.docker_mananger] Container started with ID 5bddd2bfe437. Setting up logs at ./log/docker/node1_2026-03-24_04-39-42__958d3707-b578-42b6-8e02-7ce9eec4bb6a__wakuorg_nwaku:latest.log
2026-03-24 04:39:42.857 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 13621
2026-03-24 04:39:42.857 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-24 04:39:43.379 ERROR [src.node.docker_mananger] Max retries reached for container 41228d447eb5. Exiting log stream.
2026-03-24 04:39:43.858 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13621/health" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:43.861 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-24 04:39:43.861 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-24 04:39:43.861 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13621/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:43.863 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.179.228/tcp/13622/p2p/16Uiu2HAmKjzzjjFSR1yt9Cyy5uAZi3wV9uDWHV3sXRKC9dfJ8nGH","/ip4/172.18.179.228/tcp/13623/ws/p2p/16Uiu2HAmKjzzjjFSR1yt9Cyy5uAZi3wV9uDWHV3sXRKC9dfJ8nGH"],"enrUri":"enr:-L24QE_Q2_RmfXpmRuDxcU0D8k-c5oLLxUQEJbm0ndRQ1Id9QcM6k-wRip-Ix0kzcChqqABF1snw_Pxrf9wmaWM8uwwCgmlkgnY0gmlwhKwSs-SKbXVsdGlhZGRyc5YACASsErPkBjU2AAoErBKz5AY1N90DgnJzhQACAQAAiXNlY3AyNTZrMaEDaUpxTBjwAD1mnob-bR1vRGuZOClQSQCudV3NeBWDQbKDdGNwgjU2g3VkcII1OIV3YWt1MgU"}'
2026-03-24 04:39:43.864 INFO [src.node.waku_node] REST service is ready !!
2026-03-24 04:39:43.871 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-24 04:39:43.871 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-24_04-39-42__958d3707-b578-42b6-8e02-7ce9eec4bb6a__wakuorg_nwaku:latest.log
2026-03-24 04:39:43.871 DEBUG [src.node.waku_node] Starting Node...
2026-03-24 04:39:43.871 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-24 04:39:43.872 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-24 04:39:43.872 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.52.104
2026-03-24 04:39:43.872 DEBUG [src.node.docker_mananger] Generated ports ['17604', '17605', '17606', '17607', '17608']
2026-03-24 04:39:43.873 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-24 04:39:43.873 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-24 04:39:43.873 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-24 04:39:43.873 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-24 04:39:43.873 DEBUG [src.node.waku_node] Using volumes []
2026-03-24 04:39:43.873 DEBUG [src.node.docker_mananger] docker run -i -t -p 17604:17604 -p 17605:17605 -p 17606:17606 -p 17607:17607 -p 17608:17608 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=17606 --rest-port=17604 --tcp-port=17605 --discv5-udp-port=17607 --rest-address=0.0.0.0 --nat=extip:172.18.52.104 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=5cfaf5dbedba3b406c456b563cc5bf370ab065e2faabfef879dbc7ae4d38d0af --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=17608 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QE_Q2_RmfXpmRuDxcU0D8k-c5oLLxUQEJbm0ndRQ1Id9QcM6k-wRip-Ix0kzcChqqABF1snw_Pxrf9wmaWM8uwwCgmlkgnY0gmlwhKwSs-SKbXVsdGlhZGRyc5YACASsErPkBjU2AAoErBKz5AY1N90DgnJzhQACAQAAiXNlY3AyNTZrMaEDaUpxTBjwAD1mnob-bR1vRGuZOClQSQCudV3NeBWDQbKDdGNwgjU2g3VkcII1OIV3YWt1MgU
2026-03-24 04:39:44.089 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.52.104 waku 7032735a7d0251468454f3aa12b36612395220e91efc46aa09da52fabaa34751
2026-03-24 04:39:44.127 DEBUG [src.node.docker_mananger] Container started with ID 7032735a7d02. Setting up logs at ./log/docker/node2_2026-03-24_04-39-42__958d3707-b578-42b6-8e02-7ce9eec4bb6a__wakuorg_nwaku:latest.log
2026-03-24 04:39:44.128 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 17604
2026-03-24 04:39:44.128 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-24 04:39:45.128 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17604/health" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.131 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-24 04:39:45.132 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-24 04:39:45.132 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17604/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.134 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.52.104/tcp/17605/p2p/16Uiu2HAmSatezV3A8ok1dCMrzii8d5kbfK8JpQJ47RJ1L485aUFp","/ip4/172.18.52.104/tcp/17606/ws/p2p/16Uiu2HAmSatezV3A8ok1dCMrzii8d5kbfK8JpQJ47RJ1L485aUFp"],"enrUri":"enr:-L24QIqmoLzd3EMl1_MLobvrlOxd8bCe5vUJBFm1ReuHDaSxdeQ7mC33ElJdXciztkQ1F7ldvlHFsIwIfnGSZGtlZ8ACgmlkgnY0gmlwhKwSNGiKbXVsdGlhZGRyc5YACASsEjRoBkTFAAoErBI0aAZExt0DgnJzhQACAQAAiXNlY3AyNTZrMaEDzvbax7vWxTgiQqS1VM8k6mdZ6NW7SwrsmISSxyxHZS-DdGNwgkTFg3VkcIJEx4V3YWt1MgE"}'
2026-03-24 04:39:45.135 INFO [src.node.waku_node] REST service is ready !!
2026-03-24 04:39:45.135 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:17604/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.179.228/tcp/13622/p2p/16Uiu2HAmKjzzjjFSR1yt9Cyy5uAZi3wV9uDWHV3sXRKC9dfJ8nGH"]'
2026-03-24 04:39:45.176 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:45.179 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13621/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-24 04:39:45.185 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:45.185 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:17604/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0", "/waku/2/rs/2/1", "/waku/2/rs/2/2", "/waku/2/rs/2/3", "/waku/2/rs/2/4", "/waku/2/rs/2/5", "/waku/2/rs/2/6", "/waku/2/rs/2/7"]'
2026-03-24 04:39:45.189 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:45.190 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:45.199 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:45.200 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:45.301 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:45.301 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.304 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185190020244,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:45.306 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:45.306 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17604/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.308 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185190020244,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:45.310 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:45.317 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:45.318 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:45.419 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:45.419 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.422 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185310260159,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:45.424 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:45.424 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17604/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.426 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185310260159,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:45.428 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:45.435 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:45.436 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:45.537 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:45.537 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.540 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185428300634,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:45.542 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:45.542 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17604/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F2" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.544 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185428300634,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:45.547 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F3" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:45.555 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:45.555 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:45.656 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:45.657 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F3" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.660 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185547769806,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:45.661 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:45.661 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17604/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F3" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.664 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185547769806,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:45.665 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F4" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:45.673 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:45.673 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:45.773 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:45.774 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F4" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.777 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185665673793,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:45.778 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:45.779 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17604/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F4" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.781 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185665673793,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:45.782 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F5" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:45.790 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:45.791 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:45.891 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:45.892 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F5" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.894 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185782857446,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:45.896 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:45.896 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17604/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F5" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:45.900 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185782857446,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:45.902 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F6" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:45.910 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:45.910 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:46.011 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:46.012 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F6" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:46.014 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185902104167,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:46.016 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:46.016 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17604/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F6" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:46.018 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327185902104167,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:46.020 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F7" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-24 04:39:46.027 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-24 04:39:46.028 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-24 04:39:46.128 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:46.129 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13621/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F7" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:46.131 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327186020022103,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:46.133 DEBUG [src.steps.sharding] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-24 04:39:46.133 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17604/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F7" -H "Content-Type: application/json" -d 'None'
2026-03-24 04:39:46.136 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774327186020022103,"ephemeral":false,"proof":""}]'
2026-03-24 04:39:46.138 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-24 04:39:46.139 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-24 04:39:46.140 DEBUG [src.node.waku_node] Stopping container with id 5bddd2bfe437
2026-03-24 04:39:46.724 DEBUG [src.node.waku_node] Container stopped.
2026-03-24 04:39:46.724 DEBUG [src.node.waku_node] Stopping container with id 7032735a7d02
2026-03-24 04:39:47.304 DEBUG [src.node.waku_node] Container stopped.
2026-03-24 04:39:47.306 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-24 04:39:47.336 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-24 04:39:47.373 DEBUG [src.node.docker_mananger] No errors found in the waku logs.