209 lines
35 KiB
Plaintext

2026-01-30 04:37:43.818 DEBUG [tests.conftest] Running fixture setup: test_id
2026-01-30 04:37:43.818 DEBUG [tests.conftest] Running test: test_node_without_sync_or_relay_stays_empty with id: 2026-01-30_04-37-43__19fd5f90-415e-42a4-97c7-693a0f82f533
2026-01-30 04:37:43.818 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-01-30 04:37:43.825 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-30 04:37:43.826 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-01-30_04-37-43__19fd5f90-415e-42a4-97c7-693a0f82f533__wakuorg_nwaku:latest.log
2026-01-30 04:37:43.832 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-30 04:37:43.832 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-01-30_04-37-43__19fd5f90-415e-42a4-97c7-693a0f82f533__wakuorg_nwaku:latest.log
2026-01-30 04:37:43.838 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-30 04:37:43.838 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-01-30_04-37-43__19fd5f90-415e-42a4-97c7-693a0f82f533__wakuorg_nwaku:latest.log
2026-01-30 04:37:43.838 DEBUG [src.steps.store] Running fixture setup: store_setup
2026-01-30 04:37:43.839 DEBUG [src.node.waku_node] Starting Node...
2026-01-30 04:37:43.839 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-30 04:37:43.841 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-30 04:37:43.841 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.113.9
2026-01-30 04:37:43.841 DEBUG [src.node.docker_mananger] Generated ports ['35978', '35979', '35980', '35981', '35982']
2026-01-30 04:37:43.841 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-30 04:37:43.841 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-30 04:37:43.842 DEBUG [src.node.waku_node] Using volumes []
2026-01-30 04:37:43.842 DEBUG [src.node.docker_mananger] docker run -i -t -p 35978:35978 -p 35979:35979 -p 35980:35980 -p 35981:35981 -p 35982:35982 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=35980 --rest-port=35978 --tcp-port=35979 --discv5-udp-port=35981 --rest-address=0.0.0.0 --nat=extip:172.18.113.9 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=6bab7fed2c65cfc2a91d12eae14c0374b659ddf778ea73e08e32fe77efc8df7f --store-sync=true --store=true --store-sync-range=300 --store-sync-interval=6 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=35982 --metrics-logging=true --store-sync-relay-jitter=0 --relay=true --dns-discovery=false
2026-01-30 04:37:44.022 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.113.9 waku bc51d53e79d7b0faa70f41ea034cbf0b69edb55e13159a9e3cafc99992fb42b7
2026-01-30 04:37:44.053 DEBUG [src.node.docker_mananger] Container started with ID bc51d53e79d7. Setting up logs at ./log/docker/node1_2026-01-30_04-37-43__19fd5f90-415e-42a4-97c7-693a0f82f533__wakuorg_nwaku:latest.log
2026-01-30 04:37:44.055 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 35978
2026-01-30 04:37:44.055 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-30 04:37:44.060 ERROR [src.node.docker_mananger] Max retries reached for container 323d731177ff. Exiting log stream.
2026-01-30 04:37:44.625 ERROR [src.node.docker_mananger] Max retries reached for container 181bdc4571f5. Exiting log stream.
2026-01-30 04:37:45.056 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35978/health" -H "Content-Type: application/json" -d 'None'
2026-01-30 04:37:45.059 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-30 04:37:45.059 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-30 04:37:45.059 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35978/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-30 04:37:45.061 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.113.9/tcp/35979/p2p/16Uiu2HAmKLpKkzmexp1uubq7nmC7YpcQMz2cSCL7LGq6w51w4ZoY","/ip4/172.18.113.9/tcp/35980/ws/p2p/16Uiu2HAmKLpKkzmexp1uubq7nmC7YpcQMz2cSCL7LGq6w51w4ZoY"],"enrUri":"enr:-L24QMMFJl8SgKo7DL5F1Xg_7sO29J2eywKXInImWiCBRntDJyzw0yWjSWSZisu3n0SSeWK0rrvSvcTELt6hziuiGokCgmlkgnY0gmlwhKwScQmKbXVsdGlhZGRyc5YACASsEnEJBoyLAAoErBJxCQaMjN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDY1oEXUHrHXZeoSxyVyWdsMWlgQbD-51Qws5BiLjLx-ODdGNwgoyLg3VkcIKMjYV3YWt1MhM"}'
2026-01-30 04:37:45.061 INFO [src.node.waku_node] REST service is ready !!
2026-01-30 04:37:45.062 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:45.062 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:45.077 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:45.077 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:45.288 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:45.288 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:45.292 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:45.292 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:45.503 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:45.503 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:45.507 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:45.508 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:45.718 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:45.719 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:45.723 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:45.723 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:45.933 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:45.934 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:45.938 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:45.938 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:46.149 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:46.149 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:46.155 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:46.155 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:46.366 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:46.366 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:46.371 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:46.371 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:46.581 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:46.582 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:46.585 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:46.585 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:46.796 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:46.796 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:46.801 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:46.801 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:47.012 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:47.012 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:47.017 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:47.017 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:47.228 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:47.228 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:47.232 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:47.232 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:47.443 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:47.443 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:47.448 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:47.448 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:47.659 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:47.659 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:47.663 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:47.663 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:47.874 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:47.874 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:47.878 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:47.878 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:48.089 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:48.089 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:48.093 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:48.093 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:48.304 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:48.304 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:48.308 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:48.308 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:48.519 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:48.519 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:48.523 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:48.523 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:48.734 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:48.734 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:48.738 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:48.738 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:48.949 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:48.949 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:48.953 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:48.954 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:49.164 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:49.165 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:49.169 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:49.169 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:49.379 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:49.380 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:49.384 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:49.384 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:49.595 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:49.595 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:49.600 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:49.600 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:49.811 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:49.811 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:49.815 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:49.815 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:50.026 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:50.026 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:50.030 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:50.031 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:50.241 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:50.242 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:50.246 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:50.246 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:50.456 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:50.457 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:50.461 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:50.461 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:50.672 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:50.672 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:50.676 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:50.676 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:50.887 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:50.888 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:50.892 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:50.892 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:51.103 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:51.103 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:51.107 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:51.107 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:51.318 DEBUG [src.steps.store] Relaying message
2026-01-30 04:37:51.318 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35978/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-01-30 04:37:51.322 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:37:51.322 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-01-30 04:37:51.533 DEBUG [src.node.waku_node] Starting Node...
2026-01-30 04:37:51.533 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-30 04:37:51.535 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-30 04:37:51.535 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.230.71
2026-01-30 04:37:51.535 DEBUG [src.node.docker_mananger] Generated ports ['65431', '65432', '65433', '65434', '65435']
2026-01-30 04:37:51.535 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-30 04:37:51.535 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-30 04:37:51.536 DEBUG [src.node.waku_node] Using volumes []
2026-01-30 04:37:51.536 DEBUG [src.node.docker_mananger] docker run -i -t -p 65431:65431 -p 65432:65432 -p 65433:65433 -p 65434:65434 -p 65435:65435 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=65433 --rest-port=65431 --tcp-port=65432 --discv5-udp-port=65434 --rest-address=0.0.0.0 --nat=extip:172.18.230.71 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=c6849fa1de345bb8d96fb7ace84b7fdceb0c8d2c9774acd0edb7cce99fdd6daa --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=65435 --metrics-logging=true --relay=false --dns-discovery=false --discv5-bootstrap-node=enr:-L24QMMFJl8SgKo7DL5F1Xg_7sO29J2eywKXInImWiCBRntDJyzw0yWjSWSZisu3n0SSeWK0rrvSvcTELt6hziuiGokCgmlkgnY0gmlwhKwScQmKbXVsdGlhZGRyc5YACASsEnEJBoyLAAoErBJxCQaMjN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDY1oEXUHrHXZeoSxyVyWdsMWlgQbD-51Qws5BiLjLx-ODdGNwgoyLg3VkcIKMjYV3YWt1MhM
2026-01-30 04:37:51.720 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.230.71 waku 2936fb0f81c328067daa4a85b446d180920b6d7898ecdf61d6af998c88e32007
2026-01-30 04:37:51.748 DEBUG [src.node.docker_mananger] Container started with ID 2936fb0f81c3. Setting up logs at ./log/docker/node2_2026-01-30_04-37-43__19fd5f90-415e-42a4-97c7-693a0f82f533__wakuorg_nwaku:latest.log
2026-01-30 04:37:51.749 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 65431
2026-01-30 04:37:51.749 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-30 04:37:52.749 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:65431/health" -H "Content-Type: application/json" -d 'None'
2026-01-30 04:37:52.753 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-30 04:37:52.753 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-30 04:37:52.753 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:65431/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-30 04:37:52.756 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.230.71/tcp/65432/p2p/16Uiu2HAm7gJ92o4jpHZYgkMmY7VQQB9xjB7LEtHffU21xEm1Ch9k","/ip4/172.18.230.71/tcp/65433/ws/p2p/16Uiu2HAm7gJ92o4jpHZYgkMmY7VQQB9xjB7LEtHffU21xEm1Ch9k"],"enrUri":"enr:-L24QAuL2t9cM7cJZ_DOGe-9UQamOUJNTwKgwfKgQ1KCJnFPbbYpX0iZhbb00Waci3txONN5ZS4XsouOYioI461AwPACgmlkgnY0gmlwhKwS5keKbXVsdGlhZGRyc5YACASsEuZHBv-YAAoErBLmRwb_md0DgnJzhQADAQAAiXNlY3AyNTZrMaECtgtChu6yFjNw-e1Q8G6Vym0_nW4Ygv6xfSe3F2Pz75ODdGNwgv-Yg3VkcIL_moV3YWt1MgA"}'
2026-01-30 04:37:52.756 INFO [src.node.waku_node] REST service is ready !!
2026-01-30 04:37:52.756 DEBUG [src.node.waku_node] Starting Node...
2026-01-30 04:37:52.756 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-30 04:37:52.758 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-30 04:37:52.758 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.104.81
2026-01-30 04:37:52.758 DEBUG [src.node.docker_mananger] Generated ports ['14133', '14134', '14135', '14136', '14137']
2026-01-30 04:37:52.759 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-30 04:37:52.759 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-30 04:37:52.759 DEBUG [src.node.waku_node] Using volumes []
2026-01-30 04:37:52.759 DEBUG [src.node.docker_mananger] docker run -i -t -p 14133:14133 -p 14134:14134 -p 14135:14135 -p 14136:14136 -p 14137:14137 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=14135 --rest-port=14133 --tcp-port=14134 --discv5-udp-port=14136 --rest-address=0.0.0.0 --nat=extip:172.18.104.81 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=ba6a6add3f4da6197b68cae9b5fadadda7aa95abc7e3dfaa5dd8bd5349bdf0f3 --store-sync=true --store=true --store-sync-range=300 --store-sync-interval=6 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=14137 --metrics-logging=true --store-sync-relay-jitter=0 --relay=false --dns-discovery=false --discv5-bootstrap-node=enr:-L24QMMFJl8SgKo7DL5F1Xg_7sO29J2eywKXInImWiCBRntDJyzw0yWjSWSZisu3n0SSeWK0rrvSvcTELt6hziuiGokCgmlkgnY0gmlwhKwScQmKbXVsdGlhZGRyc5YACASsEnEJBoyLAAoErBJxCQaMjN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDY1oEXUHrHXZeoSxyVyWdsMWlgQbD-51Qws5BiLjLx-ODdGNwgoyLg3VkcIKMjYV3YWt1MhM
2026-01-30 04:37:52.942 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.104.81 waku 7148cbf8f12d1f10adc54bfa1708165c878e4560f582ba29e03222a85f4abdad
2026-01-30 04:37:52.972 DEBUG [src.node.docker_mananger] Container started with ID 7148cbf8f12d. Setting up logs at ./log/docker/node3_2026-01-30_04-37-43__19fd5f90-415e-42a4-97c7-693a0f82f533__wakuorg_nwaku:latest.log
2026-01-30 04:37:52.973 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 14133
2026-01-30 04:37:52.975 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-30 04:37:53.976 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:14133/health" -H "Content-Type: application/json" -d 'None'
2026-01-30 04:37:53.980 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-30 04:37:53.980 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-30 04:37:53.980 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:14133/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-30 04:37:53.983 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.104.81/tcp/14134/p2p/16Uiu2HAm4EMuZezJkqydnFzwcFZfP71Xiy7i6xWvu5hAYwx7rcJP","/ip4/172.18.104.81/tcp/14135/ws/p2p/16Uiu2HAm4EMuZezJkqydnFzwcFZfP71Xiy7i6xWvu5hAYwx7rcJP"],"enrUri":"enr:-L24QDcmzoldImqmDEVzia0Hzk87K_ZLZw-0S6ZDi8NEz1KUcAyVvIrHs7OC1abjnKyWNMIQ1G5bsuFwQf5tMSTpLxoCgmlkgnY0gmlwhKwSaFGKbXVsdGlhZGRyc5YACASsEmhRBjc2AAoErBJoUQY3N90DgnJzhQADAQAAiXNlY3AyNTZrMaECgtNbuowkss6yWnS7LD0fe-6D3CoJDYFrA7KdUaYJFUSDdGNwgjc2g3VkcII3OIV3YWt1MhI"}'
2026-01-30 04:37:53.983 INFO [src.node.waku_node] REST service is ready !!
2026-01-30 04:37:53.983 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:14133/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.113.9/tcp/35979/p2p/16Uiu2HAmKLpKkzmexp1uubq7nmC7YpcQMz2cSCL7LGq6w51w4ZoY"]'
2026-01-30 04:37:54.015 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-30 04:38:12.015 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:14133/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F0&cursor=&pageSize=200&ascending=true&peer_id=" -H "Content-Type: application/json" -d 'None'
2026-01-30 04:38:12.020 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x2cb167c659e804ab9d3a2949b4e60f45b3d7adf62f96105cad856a8a385dae9c"},{"messageHash":"0x660999f7236db8919a5aea5d764ee8cf0bb6e0c73088c720ac8796ff810136d4"},{"messageHash":"0x6564291941faa8c65e65d90610bf5d74050e9f4c42563f78ef7442a6f25909c0"},{"messageHash":"0x88114a6e869d91b5135b797ef9a1661bf1080a6c6c74ff2631e5a60957a5e99f"},{"messageHash":"0x853fa224e42681a68d33f5e4f02a780aa6054c713523bc56f6073fb59d220fd4"},{"messageHash":"0x0d90318e40164530ddceb518ed7e2222d683c4160368940b4ecdcb54c147bc7b"},{"messageHash":"0x174e06f15cb9583b08dc1d732842498fd33ddd3439d5ba20f56a2647b48c5b56"},{"messageHash":"0x3c1f94b42b28fa0fb1ead28023f8f2b19e3bfc140808515139dc6321297c94c9"},{"messageHash":"0xda640e431c310c6d8504544930e1924b68c6122ddd966d29845b9ebf0fac8a5c"},{"messageHash":"0x58a620b02cba793d003985214ee97f667d6cdc3f010600617e5ecc3476f7b885"},{"messageHash":"0xab84101748a628e87b04afd101c0b375600b630a71d54de688f207cdeb582fb2"},{"messageHash":"0xcaf9c6d2c10e70d844e880374e813c5d4d44aeec8de556034715b19d7f34ec91"},{"messageHash":"0x555b4cf2423a896c4c2df6f4f0ce8229d0300775a4fee195bf3cda5b22fb1f00"},{"messageHash":"0x23f326873b9cc2cad6c36e5dffd5c668a3ab4d4cea00dae510221f44a102efc4"},{"messageHash":"0x144b9fc36b0dc1c3c9233b280dcb8ecb3bda87a873762c5f1b7546a579d54398"},{"messageHash":"0xe00efa08605020e313d4899f146e75951f713be41343c3319a16f790fd7d1102"},{"messageHash":"0x0f185adab3024824260ca28de3655ec7653ef8062c75b9e2c96dd520eb9fbbdf"},{"messageHash":"0x939b9857acc0570ddb01308047bff0e5342782306374da3b6ac59bd2c40bb871"},{"messageHash":"0xf33461e6e891f85b5474af1133fd8b7324ef881de54eccfb83da902cb3efa110"},{"messageHash":"0xd8bb04a1ac1171634141d4189b3a84ea7f9160a9a9e3d109bb88e6a69a09eca4"},{"messageHash":"0xd2d0b206631e4d9c5b51aa51a47b50a7537203a9f23850529e72afa77a2a799d"},{"messageHash":"0x013f525a8d37068eca01f0ed3a9724111784ce591af1c32c5b1e3cc8fd63fb7b"},{"messageHash":"0xef1d770d75d6e257c4c190b17e827f30d02393ecb859004d5ff5baa8091efc0e"},{"messageHash":"0x2e756eadda64ec0ca889ff948faec86e9ab8066ff1ac31aa9a2933421922e499"},{"messageHash":"0x69f330f566b798ad401b26e1bebefe8f34955752f2090889348d3d6a76acf38d"},{"messageHash":"0xe89ffd641ee844e7646b389693b1f2af391bd5fff2149009e2179630e288b322"},{"messageHash":"0xe51b350621d3ec65c332026330d901def3bc595a8d3ec18d7d459c1796a9502b"},{"messageHash":"0x9269f33be35527d170f9fa5637532a2ab426ab26960b318ff451eb8ce1be248b"},{"messageHash":"0x4bb0e9dd72dff566ae5d009d7a429674e2e20a10e6f07288f43f41b092aff2d9"},{"messageHash":"0xdf73fb1fec7aa0701dd0f9e411ee5f7c484d861ffaf0d8bd455195907f5aaf94"}]}'
2026-01-30 04:38:12.020 DEBUG [tests.store_sync.test_store_sync] Node3 store has 30 messages expected 30
2026-01-30 04:38:12.022 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-01-30 04:38:12.023 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-01-30 04:38:12.023 DEBUG [src.node.waku_node] Stopping container with id bc51d53e79d7
2026-01-30 04:38:12.626 DEBUG [src.node.waku_node] Container stopped.
2026-01-30 04:38:12.626 DEBUG [src.node.waku_node] Stopping container with id 2936fb0f81c3
2026-01-30 04:38:13.183 DEBUG [src.node.waku_node] Container stopped.
2026-01-30 04:38:13.184 DEBUG [src.node.waku_node] Stopping container with id 7148cbf8f12d
2026-01-30 04:38:13.438 ERROR [src.node.docker_mananger] Max retries reached for container bc51d53e79d7. Exiting log stream.
2026-01-30 04:38:13.996 ERROR [src.node.docker_mananger] Max retries reached for container 2936fb0f81c3. Exiting log stream.
2026-01-30 04:38:23.712 DEBUG [src.node.waku_node] Container stopped.
2026-01-30 04:38:23.714 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-01-30 04:38:23.731 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-30 04:38:23.732 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-30 04:38:23.744 DEBUG [src.node.docker_mananger] No errors found in the waku logs.