DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id DEBUG  tests.conftest:conftest.py:57 Running test: test_get_store_message_with_meta with id: 2025-12-10_04-12-54__76d9c9e5-33d4-4061-b29c-d6a0acf45cac DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup DEBUG  src.steps.store:store.py:31 Running fixture setup: store_setup DEBUG  src.steps.store:store.py:39 Running fixture setup: node_setup DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/publishing_node1_2025-12-10_04-12-54__76d9c9e5-33d4-4061-b29c-d6a0acf45cac__wakuorg_nwaku:latest.log DEBUG  src.node.waku_node:waku_node.py:90 Starting Node... DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.213.172 DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['20084', '20085', '20086', '20087', '20088'] DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN DEBUG  src.node.waku_node:waku_node.py:178 Using volumes [] DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 20084:20084 -p 20085:20085 -p 20086:20086 -p 20087:20087 -p 20088:20088 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=20086 --rest-port=20084 --tcp-port=20085 --discv5-udp-port=20087 --rest-address=0.0.0.0 --nat=extip:172.18.213.172 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=f64b2eede652a0fba68bd053f0b60ac0f7830c66cfb467dad9fcf926c40fe1fa --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=20088 --metrics-logging=true --store=true --relay=true ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container efcd0bf84575. Exiting log stream. DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.213.172 waku 774b3faf0ab1b86182b2c02c7ec9eb2d0394aa80a2bf954328d9798296ad71cf DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 774b3faf0ab1. Setting up logs at ./log/docker/publishing_node1_2025-12-10_04-12-54__76d9c9e5-33d4-4061-b29c-d6a0acf45cac__wakuorg_nwaku:latest.log DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 20084 DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 551c78d3e78a. Exiting log stream. INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:20084/health" -H "Content-Type: application/json" -d 'None' INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}' INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !! INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:20084/debug/v1/info" -H "Content-Type: application/json" -d 'None' INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.213.172/tcp/20085/p2p/16Uiu2HAm5HUuS5JDUtF1jbgeYQK8jAH2dyKmpY4hCR9rKWyq5iZp","/ip4/172.18.213.172/tcp/20086/ws/p2p/16Uiu2HAm5HUuS5JDUtF1jbgeYQK8jAH2dyKmpY4hCR9rKWyq5iZp"],"enrUri":"enr:-L24QJOJkUVd-K1qA7UKo5pcQlP4zDNJfWC_2bYm1QG62aGLWcxuVUkIo3JloWKi4xjp8ZjT-jb9RIdBoK4-B5we8skCgmlkgnY0gmlwhKwS1ayKbXVsdGlhZGRyc5YACASsEtWsBk51AAoErBLVrAZOdt0DgnJzhQADAQAAiXNlY3AyNTZrMaECknuzNjZ40LqKiSrtGjL_tBw77cZpnr0h5rK9YzPd5BODdGNwgk51g3VkcIJOd4V3YWt1MgM"}' INFO  src.node.waku_node:waku_node.py:292 REST service is ready !! DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/store_node1_2025-12-10_04-12-54__76d9c9e5-33d4-4061-b29c-d6a0acf45cac__wakuorg_nwaku:latest.log DEBUG  src.node.waku_node:waku_node.py:90 Starting Node... DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.49.255 DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['42551', '42552', '42553', '42554', '42555'] DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN DEBUG  src.node.waku_node:waku_node.py:178 Using volumes [] DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 42551:42551 -p 42552:42552 -p 42553:42553 -p 42554:42554 -p 42555:42555 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=42553 --rest-port=42551 --tcp-port=42552 --discv5-udp-port=42554 --rest-address=0.0.0.0 --nat=extip:172.18.49.255 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=83b0d5185be47400a7dacffcd018bdeb87d6eaa857af97eb91c15dd75dc1e1bc --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=42555 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QJOJkUVd-K1qA7UKo5pcQlP4zDNJfWC_2bYm1QG62aGLWcxuVUkIo3JloWKi4xjp8ZjT-jb9RIdBoK4-B5we8skCgmlkgnY0gmlwhKwS1ayKbXVsdGlhZGRyc5YACASsEtWsBk51AAoErBLVrAZOdt0DgnJzhQADAQAAiXNlY3AyNTZrMaECknuzNjZ40LqKiSrtGjL_tBw77cZpnr0h5rK9YzPd5BODdGNwgk51g3VkcIJOd4V3YWt1MgM --storenode=/ip4/172.18.213.172/tcp/20085/p2p/16Uiu2HAm5HUuS5JDUtF1jbgeYQK8jAH2dyKmpY4hCR9rKWyq5iZp --store=true --relay=true DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.49.255 waku 58b4b4303d1483cc27dad0ed2f161ced94db0ea056f2f3749e8b9e548a8e192e DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 58b4b4303d14. Setting up logs at ./log/docker/store_node1_2025-12-10_04-12-54__76d9c9e5-33d4-4061-b29c-d6a0acf45cac__wakuorg_nwaku:latest.log DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 42551 DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:42551/health" -H "Content-Type: application/json" -d 'None' INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"READY"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"READY"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}' INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !! INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:42551/debug/v1/info" -H "Content-Type: application/json" -d 'None' INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.49.255/tcp/42552/p2p/16Uiu2HAmEjsY7HRsUqX8vTBSJyY7GyJ7cJ2VRmAQnm1WvuduuLtC","/ip4/172.18.49.255/tcp/42553/ws/p2p/16Uiu2HAmEjsY7HRsUqX8vTBSJyY7GyJ7cJ2VRmAQnm1WvuduuLtC"],"enrUri":"enr:-L24QNQr_pcUMaW3ir5v6uk4CkEtcom_RWrd8SBSX7f29k7Ycj2N3aH-_cpyu0zUmKXBiDmwgWrx3BwkM4Xugk_Zy8UCgmlkgnY0gmlwhKwSMf-KbXVsdGlhZGRyc5YACASsEjH_BqY4AAoErBIx_wamOd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDHveS6qz_j7dgCK4-z9tP20RBGB23T18EltFQtq-yHaWDdGNwgqY4g3VkcIKmOoV3YWt1MgM"}' INFO  src.node.waku_node:waku_node.py:292 REST service is ready !! INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:42551/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.213.172/tcp/20085/p2p/16Uiu2HAm5HUuS5JDUtF1jbgeYQK8jAH2dyKmpY4hCR9rKWyq5iZp"]' INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK' INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:20084/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]' INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK' INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:42551/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]' INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK' DEBUG  src.steps.store:store.py:132 Relaying message INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:20084/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)', "meta": "U3RvcmUgd29ya3MhIQ=="}' INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK' DEBUG  src.libs.common:common.py:47 Sleeping for 0.2 seconds DEBUG  src.steps.store:store.py:211 Checking that peer wakuorg/nwaku:latest can find the stored messages INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:20084/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=5&ascending=true" -H "Content-Type: application/json" -d 'None' INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0xd40b46bc3d63c99b0a916e6a5caefc3ff7c63445b3c0866413da1abb17f5bc27"}]}' DEBUG  src.steps.store:store.py:228 messages length is 1 DEBUG  src.steps.store:store.py:211 Checking that peer wakuorg/nwaku:latest can find the stored messages INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:42551/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&pageSize=5&ascending=true" -H "Content-Type: application/json" -d 'None' INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0xd40b46bc3d63c99b0a916e6a5caefc3ff7c63445b3c0866413da1abb17f5bc27"}]}' DEBUG  src.steps.store:store.py:228 messages length is 1 DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 774b3faf0ab1 DEBUG  src.node.waku_node:waku_node.py:241 Container stopped. DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 58b4b4303d14 DEBUG  src.node.waku_node:waku_node.py:241 Container stopped. DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs. DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.