102 lines
18 KiB
Plaintext

2026-04-23 22:03:57.904 DEBUG [tests.conftest] Running fixture setup: test_id
2026-04-23 22:03:57.904 DEBUG [tests.conftest] Running test: test_store_node_paused_and_unpaused with id: 2026-04-23_22-03-57__704d541c-6073-4459-bfa9-4a26075d41ad
2026-04-23 22:03:57.904 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-04-23 22:03:57.905 DEBUG [src.steps.store] Running fixture setup: store_setup
2026-04-23 22:03:57.905 DEBUG [src.steps.store] Running fixture setup: node_setup
2026-04-23 22:03:57.911 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 22:03:57.911 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/publishing_node1_2026-04-23_22-03-57__704d541c-6073-4459-bfa9-4a26075d41ad__wakuorg_nwaku:latest.log
2026-04-23 22:03:57.911 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 22:03:57.912 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 22:03:57.913 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 22:03:57.913 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.137.123
2026-04-23 22:03:57.913 DEBUG [src.node.docker_mananger] Generated ports ['27293', '27294', '27295', '27296', '27297']
2026-04-23 22:03:57.913 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 22:03:57.913 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 22:03:57.913 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 22:03:57.914 DEBUG [src.node.docker_mananger] docker run -i -t -p 27293:27293 -p 27294:27294 -p 27295:27295 -p 27296:27296 -p 27297:27297 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=27295 --rest-port=27293 --tcp-port=27294 --discv5-udp-port=27296 --rest-address=0.0.0.0 --nat=extip:172.18.137.123 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=05be6f9cfaf21a948a6b692bfec6af74ecc26ea87df57dd1cff3d33b0d1e5d0d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=27297 --metrics-logging=true --store=true --relay=true
2026-04-23 22:03:58.105 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.137.123 waku 8da0ca16ac973565006c9e1d4aec077605e26f7f0b866916e3ede97dd30d0a47
2026-04-23 22:03:58.140 DEBUG [src.node.docker_mananger] Container started with ID 8da0ca16ac97. Setting up logs at ./log/docker/publishing_node1_2026-04-23_22-03-57__704d541c-6073-4459-bfa9-4a26075d41ad__wakuorg_nwaku:latest.log
2026-04-23 22:03:58.141 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 27293
2026-04-23 22:03:58.142 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 22:03:58.229 ERROR [src.node.docker_mananger] Max retries reached for container 81ad1c7fe06f. Exiting log stream.
2026-04-23 22:03:58.694 ERROR [src.node.docker_mananger] Max retries reached for container 417abd32b1b8. Exiting log stream.
2026-04-23 22:03:59.143 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27293/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 22:03:59.146 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 22:03:59.146 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 22:03:59.146 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27293/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 22:03:59.148 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.137.123/tcp/27294/p2p/16Uiu2HAkzf2v6624SjrfAp7N938stGqJkmiQDz8eCUrQeSm2QiZ1","/ip4/172.18.137.123/tcp/27295/ws/p2p/16Uiu2HAkzf2v6624SjrfAp7N938stGqJkmiQDz8eCUrQeSm2QiZ1"],"enrUri":"enr:-L24QKJTd5AVyOTkNWT8XK91pdRVSWuh_HAj-k68BqFiakXrd2A84tZdy1q-zSAFdunHgxdZPpHi3BqNeirWVLSsfVICgmlkgnY0gmlwhKwSiXuKbXVsdGlhZGRyc5YACASsEol7BmqeAAoErBKJewZqn90DgnJzhQDGAQAAiXNlY3AyNTZrMaECTbaoXtHQI3eIrF3ejHda8LN6dTwjXUvrhTe9UEobpeyDdGNwgmqeg3VkcIJqoIV3YWt1MgM"}'
2026-04-23 22:03:59.148 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 22:03:59.155 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-04-23 22:03:59.155 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/store_node1_2026-04-23_22-03-57__704d541c-6073-4459-bfa9-4a26075d41ad__wakuorg_nwaku:latest.log
2026-04-23 22:03:59.156 DEBUG [src.node.waku_node] Starting Node...
2026-04-23 22:03:59.156 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-04-23 22:03:59.157 DEBUG [src.node.docker_mananger] Network waku already exists
2026-04-23 22:03:59.157 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.145.127
2026-04-23 22:03:59.157 DEBUG [src.node.docker_mananger] Generated ports ['53977', '53978', '53979', '53980', '53981']
2026-04-23 22:03:59.158 DEBUG [src.node.waku_node] RLN credentials were not set
2026-04-23 22:03:59.158 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-04-23 22:03:59.158 DEBUG [src.node.waku_node] Using volumes []
2026-04-23 22:03:59.158 DEBUG [src.node.docker_mananger] docker run -i -t -p 53977:53977 -p 53978:53978 -p 53979:53979 -p 53980:53980 -p 53981:53981 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=53979 --rest-port=53977 --tcp-port=53978 --discv5-udp-port=53980 --rest-address=0.0.0.0 --nat=extip:172.18.145.127 --peer-exchange=true --discv5-discovery=true --cluster-id=198 --nodekey=b7f5e428cdaf6f6da971cc619cffa5ddd37cefcbe36448c4acc6febbba6a0b7f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=53981 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QKJTd5AVyOTkNWT8XK91pdRVSWuh_HAj-k68BqFiakXrd2A84tZdy1q-zSAFdunHgxdZPpHi3BqNeirWVLSsfVICgmlkgnY0gmlwhKwSiXuKbXVsdGlhZGRyc5YACASsEol7BmqeAAoErBKJewZqn90DgnJzhQDGAQAAiXNlY3AyNTZrMaECTbaoXtHQI3eIrF3ejHda8LN6dTwjXUvrhTe9UEobpeyDdGNwgmqeg3VkcIJqoIV3YWt1MgM --storenode=/ip4/172.18.137.123/tcp/27294/p2p/16Uiu2HAkzf2v6624SjrfAp7N938stGqJkmiQDz8eCUrQeSm2QiZ1 --store=true --relay=true
2026-04-23 22:03:59.340 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.145.127 waku 1f3b589eefe721d82d8b2b5370668ab22a8b3fad0fea7a83a12c79907a0dc640
2026-04-23 22:03:59.372 DEBUG [src.node.docker_mananger] Container started with ID 1f3b589eefe7. Setting up logs at ./log/docker/store_node1_2026-04-23_22-03-57__704d541c-6073-4459-bfa9-4a26075d41ad__wakuorg_nwaku:latest.log
2026-04-23 22:03:59.373 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 53977
2026-04-23 22:03:59.374 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 22:04:00.374 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53977/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 22:04:00.377 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 22:04:00.377 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 22:04:00.378 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53977/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 22:04:00.380 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.145.127/tcp/53978/p2p/16Uiu2HAkycaXfsd7A6S162jCLaHjjittqoWPLt4S7tasco9oztBM","/ip4/172.18.145.127/tcp/53979/ws/p2p/16Uiu2HAkycaXfsd7A6S162jCLaHjjittqoWPLt4S7tasco9oztBM"],"enrUri":"enr:-L24QPcemrrmZflfunTJAnNdgT90pm2-Rc730fDGp72XJiOZLXQSDG0uwNkiAkVt-qb231tlRwAaMkEOeP0454NA6OMCgmlkgnY0gmlwhKwSkX-KbXVsdGlhZGRyc5YACASsEpF_BtLaAAoErBKRfwbS290DgnJzhQDGAQAAiXNlY3AyNTZrMaECPjn5TROdZA5c0llq4r6jNQu57QHSvo-yZe3mRBF2wEyDdGNwgtLag3VkcILS3IV3YWt1MgM"}'
2026-04-23 22:04:00.380 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 22:04:00.380 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:53977/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.137.123/tcp/27294/p2p/16Uiu2HAkzf2v6624SjrfAp7N938stGqJkmiQDz8eCUrQeSm2QiZ1"]'
2026-04-23 22:04:00.383 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 22:04:00.383 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:27293/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/0"]'
2026-04-23 22:04:00.386 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 22:04:00.386 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:53977/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/198/0"]'
2026-04-23 22:04:00.389 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 22:04:00.390 DEBUG [src.steps.store] Relaying message
2026-04-23 22:04:00.390 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:27293/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-04-23 22:04:00.395 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 22:04:00.396 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-04-23 22:04:00.596 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-04-23 22:04:00.597 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27293/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F198%2F0&pageSize=5" -H "Content-Type: application/json" -d 'None'
2026-04-23 22:04:00.600 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x70233f745d78bad63cb444564feaad5dfa3f93bc7b681e88d6baa4e988f09279"}]}'
2026-04-23 22:04:00.600 DEBUG [src.steps.store] messages length is 1
2026-04-23 22:04:00.600 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-04-23 22:04:00.601 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53977/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F198%2F0&pageSize=5" -H "Content-Type: application/json" -d 'None'
2026-04-23 22:04:00.603 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x70233f745d78bad63cb444564feaad5dfa3f93bc7b681e88d6baa4e988f09279"}]}'
2026-04-23 22:04:00.604 DEBUG [src.steps.store] messages length is 1
2026-04-23 22:04:00.604 DEBUG [src.node.waku_node] Pausing container with id 1f3b589eefe7
2026-04-23 22:04:00.613 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-04-23 22:04:01.613 DEBUG [src.node.waku_node] Unpause container with id 1f3b589eefe7
2026-04-23 22:04:01.623 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53977/health" -H "Content-Type: application/json" -d 'None'
2026-04-23 22:04:01.625 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-04-23 22:04:01.626 INFO [src.node.waku_node] Node protocols are initialized !!
2026-04-23 22:04:01.626 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53977/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-04-23 22:04:01.628 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.145.127/tcp/53978/p2p/16Uiu2HAkycaXfsd7A6S162jCLaHjjittqoWPLt4S7tasco9oztBM","/ip4/172.18.145.127/tcp/53979/ws/p2p/16Uiu2HAkycaXfsd7A6S162jCLaHjjittqoWPLt4S7tasco9oztBM"],"enrUri":"enr:-L24QPcemrrmZflfunTJAnNdgT90pm2-Rc730fDGp72XJiOZLXQSDG0uwNkiAkVt-qb231tlRwAaMkEOeP0454NA6OMCgmlkgnY0gmlwhKwSkX-KbXVsdGlhZGRyc5YACASsEpF_BtLaAAoErBKRfwbS290DgnJzhQDGAQAAiXNlY3AyNTZrMaECPjn5TROdZA5c0llq4r6jNQu57QHSvo-yZe3mRBF2wEyDdGNwgtLag3VkcILS3IV3YWt1MgM"}'
2026-04-23 22:04:01.629 INFO [src.node.waku_node] REST service is ready !!
2026-04-23 22:04:01.629 DEBUG [src.steps.store] Relaying message
2026-04-23 22:04:01.629 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:27293/relay/v1/messages/%2Fwaku%2F2%2Frs%2F198%2F0" -H "Content-Type: application/json" -d '{"payload": "U3RvcmUgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-04-23 22:04:01.634 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-04-23 22:04:01.634 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-04-23 22:04:01.835 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-04-23 22:04:01.835 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27293/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F198%2F0&pageSize=5" -H "Content-Type: application/json" -d 'None'
2026-04-23 22:04:01.838 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x70233f745d78bad63cb444564feaad5dfa3f93bc7b681e88d6baa4e988f09279"},{"messageHash":"0x47b5bfa7534db6e65e3ef34cc4ac28b8b2d056aa2bc70ea3f6a9d238c755792b"}]}'
2026-04-23 22:04:01.838 DEBUG [src.steps.store] messages length is 2
2026-04-23 22:04:01.838 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2026-04-23 22:04:01.839 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53977/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F198%2F0&pageSize=5" -H "Content-Type: application/json" -d 'None'
2026-04-23 22:04:01.841 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x70233f745d78bad63cb444564feaad5dfa3f93bc7b681e88d6baa4e988f09279"},{"messageHash":"0x47b5bfa7534db6e65e3ef34cc4ac28b8b2d056aa2bc70ea3f6a9d238c755792b"}]}'
2026-04-23 22:04:01.842 DEBUG [src.steps.store] messages length is 2
2026-04-23 22:04:01.842 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27293/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F198%2F0&pageSize=5&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-04-23 22:04:01.845 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x70233f745d78bad63cb444564feaad5dfa3f93bc7b681e88d6baa4e988f09279"},{"messageHash":"0x47b5bfa7534db6e65e3ef34cc4ac28b8b2d056aa2bc70ea3f6a9d238c755792b"}]}'
2026-04-23 22:04:01.845 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:53977/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F198%2F0&pageSize=5&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-04-23 22:04:01.848 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x70233f745d78bad63cb444564feaad5dfa3f93bc7b681e88d6baa4e988f09279"},{"messageHash":"0x47b5bfa7534db6e65e3ef34cc4ac28b8b2d056aa2bc70ea3f6a9d238c755792b"}]}'
2026-04-23 22:04:01.850 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-04-23 22:04:01.851 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-04-23 22:04:01.851 DEBUG [src.node.waku_node] Stopping container with id 8da0ca16ac97
2026-04-23 22:04:02.305 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 22:04:02.306 DEBUG [src.node.waku_node] Stopping container with id 1f3b589eefe7
2026-04-23 22:04:02.753 DEBUG [src.node.waku_node] Container stopped.
2026-04-23 22:04:02.754 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-04-23 22:04:02.762 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-04-23 22:04:02.769 DEBUG [src.node.docker_mananger] No errors found in the waku logs.