179 lines
34 KiB
Plaintext

2025-12-26 04:12:45.901 DEBUG [tests.conftest] Running fixture setup: test_id
2025-12-26 04:12:45.901 DEBUG [tests.conftest] Running test: test_multiple_edge_service_nodes_communication with id: 2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205
2025-12-26 04:12:45.901 DEBUG [src.steps.common] Running fixture setup: common_setup
2025-12-26 04:12:45.902 DEBUG [src.steps.filter] Running fixture setup: filter_setup
2025-12-26 04:12:45.902 DEBUG [src.steps.light_push] Running fixture setup: light_push_setup
2025-12-26 04:12:45.911 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-26 04:12:45.911 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:45.917 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-26 04:12:45.917 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:45.923 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-26 04:12:45.923 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:45.923 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2025-12-26 04:12:45.923 DEBUG [src.steps.store] Running fixture setup: store_setup
2025-12-26 04:12:45.929 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-26 04:12:45.930 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:45.935 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-26 04:12:45.935 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:45.941 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-26 04:12:45.941 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node6_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:45.946 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-26 04:12:45.947 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node7_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:45.952 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-26 04:12:45.952 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node8_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:45.952 DEBUG [tests.e2e.test_e2e] Start 2 edges nodes and 3 service nodes
2025-12-26 04:12:45.953 DEBUG [src.node.waku_node] Starting Node...
2025-12-26 04:12:45.953 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-26 04:12:45.995 DEBUG [src.node.docker_mananger] Network waku created
2025-12-26 04:12:45.996 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.160.129
2025-12-26 04:12:45.996 DEBUG [src.node.docker_mananger] Generated ports ['56969', '56970', '56971', '56972', '56973']
2025-12-26 04:12:45.996 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-26 04:12:45.996 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-26 04:12:45.996 DEBUG [src.node.waku_node] Using volumes []
2025-12-26 04:12:45.996 DEBUG [src.node.docker_mananger] docker run -i -t -p 56969:56969 -p 56970:56970 -p 56971:56971 -p 56972:56972 -p 56973:56973 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=56971 --rest-port=56969 --tcp-port=56970 --discv5-udp-port=56972 --rest-address=0.0.0.0 --nat=extip:172.18.160.129 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=e48c28fa7a5b77ee8a8ebabecccbb34b0b003f59ef6b76b6f3c69fc8904ebf02 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=56973 --metrics-logging=true --relay=true --store=true --lightpush=true
2025-12-26 04:12:48.954 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.160.129 waku 9a8f43011302b904373d6fcffb9f4a3c8ff44db6dce680229464589395260dc0
2025-12-26 04:12:48.992 DEBUG [src.node.docker_mananger] Container started with ID 9a8f43011302. Setting up logs at ./log/docker/node6_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:48.992 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 56969
2025-12-26 04:12:48.994 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-26 04:12:49.994 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56969/health" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:49.998 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-26 04:12:49.998 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-26 04:12:49.999 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56969/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:50.001 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.160.129/tcp/56970/p2p/16Uiu2HAm1TnwKGBZsedLCtK19nSF4EuZmk5VycFib9ta143NSRFo","/ip4/172.18.160.129/tcp/56971/ws/p2p/16Uiu2HAm1TnwKGBZsedLCtK19nSF4EuZmk5VycFib9ta143NSRFo"],"enrUri":"enr:-L24QPFT7opqt17NdsYpLyD0KKUxA1xH3M09hZQ-OF7uWVvcUXuLvMbNdAlkLzOtk7ikT4JACNRJwwuVMRSooHS_YeECgmlkgnY0gmlwhKwSoIGKbXVsdGlhZGRyc5YACASsEqCBBt6KAAoErBKggQbei90DgnJzhQADAQAAiXNlY3AyNTZrMaECWbEmWQusRv6G8k8uuqgNTASJQc-9kJOQNs7_SbMVS_KDdGNwgt6Kg3VkcILejIV3YWt1Mgs"}'
2025-12-26 04:12:50.001 INFO [src.node.waku_node] REST service is ready !!
2025-12-26 04:12:50.002 DEBUG [src.node.waku_node] Starting Node...
2025-12-26 04:12:50.002 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-26 04:12:50.003 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-26 04:12:50.003 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.139.228
2025-12-26 04:12:50.003 DEBUG [src.node.docker_mananger] Generated ports ['57532', '57533', '57534', '57535', '57536']
2025-12-26 04:12:50.004 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-26 04:12:50.004 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-26 04:12:50.004 DEBUG [src.node.waku_node] Using volumes []
2025-12-26 04:12:50.004 DEBUG [src.node.docker_mananger] docker run -i -t -p 57532:57532 -p 57533:57533 -p 57534:57534 -p 57535:57535 -p 57536:57536 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=57534 --rest-port=57532 --tcp-port=57533 --discv5-udp-port=57535 --rest-address=0.0.0.0 --nat=extip:172.18.139.228 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=aa3e61e3b471c8b72a4a833bf5015abaebf431bf469275d38d7aa61ec6b4dabf --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=57536 --metrics-logging=true --relay=false --lightpushnode=/ip4/172.18.160.129/tcp/56970/p2p/16Uiu2HAm1TnwKGBZsedLCtK19nSF4EuZmk5VycFib9ta143NSRFo --discv5-bootstrap-node=enr:-L24QPFT7opqt17NdsYpLyD0KKUxA1xH3M09hZQ-OF7uWVvcUXuLvMbNdAlkLzOtk7ikT4JACNRJwwuVMRSooHS_YeECgmlkgnY0gmlwhKwSoIGKbXVsdGlhZGRyc5YACASsEqCBBt6KAAoErBKggQbei90DgnJzhQADAQAAiXNlY3AyNTZrMaECWbEmWQusRv6G8k8uuqgNTASJQc-9kJOQNs7_SbMVS_KDdGNwgt6Kg3VkcILejIV3YWt1Mgs
2025-12-26 04:12:50.192 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.139.228 waku 404ef91c0234b591e2d46827bc5bf7b431d09ba688bc11b3e49524b471fe7bea
2025-12-26 04:12:50.221 DEBUG [src.node.docker_mananger] Container started with ID 404ef91c0234. Setting up logs at ./log/docker/node4_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:50.222 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 57532
2025-12-26 04:12:50.222 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-26 04:12:51.223 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57532/health" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:51.227 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-26 04:12:51.227 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-26 04:12:51.227 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:57532/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:51.230 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.139.228/tcp/57533/p2p/16Uiu2HAmK34MBFHH8UMjtyzRCWeCuYr71DMuEbtcvAhqYXeqrbBe","/ip4/172.18.139.228/tcp/57534/ws/p2p/16Uiu2HAmK34MBFHH8UMjtyzRCWeCuYr71DMuEbtcvAhqYXeqrbBe"],"enrUri":"enr:-L24QGRZZ4RLvJn39FhbrFovIyRGOl0gbxpi0izIDwexDYE0c6aFsv_MU-fEqDdOoD-d0Dmte9HqGMwLa2dtlkFbS3YCgmlkgnY0gmlwhKwSi-SKbXVsdGlhZGRyc5YACASsEovkBuC9AAoErBKL5Abgvt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDXs1rg98GnH9H9nGXqCnj3HB10uRuF1PzOmn2c1ksUvmDdGNwguC9g3VkcILgv4V3YWt1MgA"}'
2025-12-26 04:12:51.230 INFO [src.node.waku_node] REST service is ready !!
2025-12-26 04:12:51.230 DEBUG [src.node.waku_node] Starting Node...
2025-12-26 04:12:51.230 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-26 04:12:51.232 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-26 04:12:51.232 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.65.128
2025-12-26 04:12:51.232 DEBUG [src.node.docker_mananger] Generated ports ['59103', '59104', '59105', '59106', '59107']
2025-12-26 04:12:51.232 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-26 04:12:51.232 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-26 04:12:51.232 DEBUG [src.node.waku_node] Using volumes []
2025-12-26 04:12:51.233 DEBUG [src.node.docker_mananger] docker run -i -t -p 59103:59103 -p 59104:59104 -p 59105:59105 -p 59106:59106 -p 59107:59107 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=59105 --rest-port=59103 --tcp-port=59104 --discv5-udp-port=59106 --rest-address=0.0.0.0 --nat=extip:172.18.65.128 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=e9b34fdcdee61db4a491f7eaaa1f9c1adf2c6eeb1c8e04793fb5adaf0b9cfb2d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=59107 --metrics-logging=true --relay=true --store=true --discv5-bootstrap-node=enr:-L24QPFT7opqt17NdsYpLyD0KKUxA1xH3M09hZQ-OF7uWVvcUXuLvMbNdAlkLzOtk7ikT4JACNRJwwuVMRSooHS_YeECgmlkgnY0gmlwhKwSoIGKbXVsdGlhZGRyc5YACASsEqCBBt6KAAoErBKggQbei90DgnJzhQADAQAAiXNlY3AyNTZrMaECWbEmWQusRv6G8k8uuqgNTASJQc-9kJOQNs7_SbMVS_KDdGNwgt6Kg3VkcILejIV3YWt1Mgs
2025-12-26 04:12:51.415 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.65.128 waku b7d440516f3dfb5c1a00e4956005de00f2dc2485e29001fa5aaa2d91044b96c7
2025-12-26 04:12:51.445 DEBUG [src.node.docker_mananger] Container started with ID b7d440516f3d. Setting up logs at ./log/docker/node7_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:51.447 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 59103
2025-12-26 04:12:51.447 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-26 04:12:52.448 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59103/health" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:52.452 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-26 04:12:52.452 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-26 04:12:52.452 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59103/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:52.454 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.65.128/tcp/59104/p2p/16Uiu2HAmVpJ9TGMbfGZESWiA7KFHhT5adZ3nrCq11cjsZoFLXrw3","/ip4/172.18.65.128/tcp/59105/ws/p2p/16Uiu2HAmVpJ9TGMbfGZESWiA7KFHhT5adZ3nrCq11cjsZoFLXrw3"],"enrUri":"enr:-L24QDim0aQiMQTG9lWeIWLD2J13Md7pAJx2RyXTSHSKtKo8XqqHdoefCFwG1tZxWeEu56JmFy6M5a8ysR-wchwmxU8CgmlkgnY0gmlwhKwSQYCKbXVsdGlhZGRyc5YACASsEkGABubgAAoErBJBgAbm4d0DgnJzhQADAQAAiXNlY3AyNTZrMaED_vkG5h00hnFT_8v5RUmuDEBNfxL4rYuv1xi78c_--UKDdGNwgubgg3VkcILm4oV3YWt1MgM"}'
2025-12-26 04:12:52.454 INFO [src.node.waku_node] REST service is ready !!
2025-12-26 04:12:52.454 DEBUG [src.node.waku_node] Starting Node...
2025-12-26 04:12:52.455 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-26 04:12:52.456 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-26 04:12:52.456 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.221.85
2025-12-26 04:12:52.456 DEBUG [src.node.docker_mananger] Generated ports ['25145', '25146', '25147', '25148', '25149']
2025-12-26 04:12:52.457 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-26 04:12:52.457 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-26 04:12:52.457 DEBUG [src.node.waku_node] Using volumes []
2025-12-26 04:12:52.457 DEBUG [src.node.docker_mananger] docker run -i -t -p 25145:25145 -p 25146:25146 -p 25147:25147 -p 25148:25148 -p 25149:25149 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=25147 --rest-port=25145 --tcp-port=25146 --discv5-udp-port=25148 --rest-address=0.0.0.0 --nat=extip:172.18.221.85 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=4d97baf6f4bdd370abd6d78ce1969b04ecbaaaceefeeb6957a2bfc57d1a6c5ca --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=25149 --metrics-logging=true --relay=true --filter=true --storenode=/ip4/172.18.65.128/tcp/59104/p2p/16Uiu2HAmVpJ9TGMbfGZESWiA7KFHhT5adZ3nrCq11cjsZoFLXrw3 --discv5-bootstrap-node=enr:-L24QDim0aQiMQTG9lWeIWLD2J13Md7pAJx2RyXTSHSKtKo8XqqHdoefCFwG1tZxWeEu56JmFy6M5a8ysR-wchwmxU8CgmlkgnY0gmlwhKwSQYCKbXVsdGlhZGRyc5YACASsEkGABubgAAoErBJBgAbm4d0DgnJzhQADAQAAiXNlY3AyNTZrMaED_vkG5h00hnFT_8v5RUmuDEBNfxL4rYuv1xi78c_--UKDdGNwgubgg3VkcILm4oV3YWt1MgM
2025-12-26 04:12:52.677 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.221.85 waku aa33cc8eecbff0d3548530bd8ff3d9acda402f0b806b05e6475fd95e0930652e
2025-12-26 04:12:52.707 DEBUG [src.node.docker_mananger] Container started with ID aa33cc8eecbf. Setting up logs at ./log/docker/node8_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:52.707 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 25145
2025-12-26 04:12:52.707 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-26 04:12:53.708 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:25145/health" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:53.721 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"READY"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-26 04:12:53.721 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-26 04:12:53.722 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:25145/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:53.735 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.221.85/tcp/25146/p2p/16Uiu2HAmVSbFTA9TUNhCf2Yfy6J6Nd7eYxqAQRfsUN4q3EDcgUSk","/ip4/172.18.221.85/tcp/25147/ws/p2p/16Uiu2HAmVSbFTA9TUNhCf2Yfy6J6Nd7eYxqAQRfsUN4q3EDcgUSk"],"enrUri":"enr:-L24QN3ToGVvRfDnDrEj3k-d6mj1h1nxwTG5YM194hbGgUEkLnZcTkbYSB9cbZ82lm-3Wbde6Oa6XF3Mkh3j6oZcHoICgmlkgnY0gmlwhKwS3VWKbXVsdGlhZGRyc5YACASsEt1VBmI6AAoErBLdVQZiO90DgnJzhQADAQAAiXNlY3AyNTZrMaED-WmWigjzSkpafTgyLruPmCxaxjk-2-MB16BigxEpoamDdGNwgmI6g3VkcIJiPIV3YWt1MgU"}'
2025-12-26 04:12:53.735 INFO [src.node.waku_node] REST service is ready !!
2025-12-26 04:12:53.736 DEBUG [src.node.waku_node] Starting Node...
2025-12-26 04:12:53.738 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-26 04:12:53.741 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-26 04:12:53.741 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.97.218
2025-12-26 04:12:53.742 DEBUG [src.node.docker_mananger] Generated ports ['26609', '26610', '26611', '26612', '26613']
2025-12-26 04:12:53.742 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-26 04:12:53.742 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-26 04:12:53.742 DEBUG [src.node.waku_node] Using volumes []
2025-12-26 04:12:53.742 DEBUG [src.node.docker_mananger] docker run -i -t -p 26609:26609 -p 26610:26610 -p 26611:26611 -p 26612:26612 -p 26613:26613 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=26611 --rest-port=26609 --tcp-port=26610 --discv5-udp-port=26612 --rest-address=0.0.0.0 --nat=extip:172.18.97.218 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=d714ddef6c2dba457b5f5e09ecfcfc5db8c9681c42b4916fccef707ced058a63 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=26613 --metrics-logging=true --relay=false --filternode=/ip4/172.18.221.85/tcp/25146/p2p/16Uiu2HAmVSbFTA9TUNhCf2Yfy6J6Nd7eYxqAQRfsUN4q3EDcgUSk --storenode=/ip4/172.18.65.128/tcp/59104/p2p/16Uiu2HAmVpJ9TGMbfGZESWiA7KFHhT5adZ3nrCq11cjsZoFLXrw3 --discv5-bootstrap-node=enr:-L24QDim0aQiMQTG9lWeIWLD2J13Md7pAJx2RyXTSHSKtKo8XqqHdoefCFwG1tZxWeEu56JmFy6M5a8ysR-wchwmxU8CgmlkgnY0gmlwhKwSQYCKbXVsdGlhZGRyc5YACASsEkGABubgAAoErBJBgAbm4d0DgnJzhQADAQAAiXNlY3AyNTZrMaED_vkG5h00hnFT_8v5RUmuDEBNfxL4rYuv1xi78c_--UKDdGNwgubgg3VkcILm4oV3YWt1MgM
2025-12-26 04:12:53.938 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.97.218 waku 5245adb380aac92b75463ca7885147cf355700c8a1b65e8128b47a8e13275b52
2025-12-26 04:12:53.967 DEBUG [src.node.docker_mananger] Container started with ID 5245adb380aa. Setting up logs at ./log/docker/node5_2025-12-26_04-12-45__229734f5-0805-4c61-a7fc-f17211120205__wakuorg_nwaku:latest.log
2025-12-26 04:12:53.968 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 26609
2025-12-26 04:12:53.969 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-26 04:12:54.969 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:26609/health" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:54.973 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"READY"},{"Filter Client":"READY"}]}'
2025-12-26 04:12:54.973 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-26 04:12:54.974 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:26609/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:54.976 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.97.218/tcp/26610/p2p/16Uiu2HAmLS1qgiL8Pn6XYtK5ncJp1ERuwKJBvbxj3Fn88jpu2T7n","/ip4/172.18.97.218/tcp/26611/ws/p2p/16Uiu2HAmLS1qgiL8Pn6XYtK5ncJp1ERuwKJBvbxj3Fn88jpu2T7n"],"enrUri":"enr:-L24QP8o5c4w4mX3e1Qu5vR7yvJEXWBOuTKowIOIH3k0nFNJIJH1CTNJRVDrgYrjOQorImO7Nwr91ZsTF4jdeNnpqDsCgmlkgnY0gmlwhKwSYdqKbXVsdGlhZGRyc5YACASsEmHaBmfyAAoErBJh2gZn890DgnJzhQADAQAAiXNlY3AyNTZrMaEDc4qhEQtAsTzGtDPtjdYDgtGTU4Ll95SZ_5cSBz5hYfmDdGNwgmfyg3VkcIJn9IV3YWt1MgA"}'
2025-12-26 04:12:54.976 INFO [src.node.waku_node] REST service is ready !!
2025-12-26 04:12:54.976 DEBUG [tests.e2e.test_e2e] Connect 3 service nodes to relay subscriptions
2025-12-26 04:12:54.977 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56969/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2025-12-26 04:12:54.982 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-26 04:12:54.984 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:59103/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2025-12-26 04:12:54.990 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-26 04:12:54.991 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:25145/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2025-12-26 04:12:54.997 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-26 04:12:55.001 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56969/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:55.004 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.65.128/tcp/46468/p2p/16Uiu2HAmVpJ9TGMbfGZESWiA7KFHhT5adZ3nrCq11cjsZoFLXrw3","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-124-g96196a","origin":"Discv5"}]'
2025-12-26 04:12:55.004 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:59103/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:55.006 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.17.0.5/tcp/37054/p2p/16Uiu2HAmVSbFTA9TUNhCf2Yfy6J6Nd7eYxqAQRfsUN4q3EDcgUSk","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-subscribe/2.0.0-beta1","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-124-g96196a","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.160.129/tcp/56970/p2p/16Uiu2HAm1TnwKGBZsedLCtK19nSF4EuZmk5VycFib9ta143NSRFo","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/lightpush/3.0.0","/vac/waku/lightpush/2.0.0-beta1","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-124-g96196a","origin":"Discv5"}]'
2025-12-26 04:12:55.006 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:25145/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:12:55.008 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.65.128/tcp/59104/p2p/16Uiu2HAmVpJ9TGMbfGZESWiA7KFHhT5adZ3nrCq11cjsZoFLXrw3","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/store-query/3.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-124-g96196a","origin":"Discv5"}]'
2025-12-26 04:12:55.009 DEBUG [src.libs.common] Sleeping for 30 seconds
2025-12-26 04:13:25.009 DEBUG [tests.e2e.test_e2e] Edge node2 makes filter subscription to pubsubtopic /waku/2/rs/3/1 and content topic /test/1/waku-filter/proto
2025-12-26 04:13:25.009 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:26609/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "1", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2025-12-26 04:13:25.048 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"1","statusDesc":"OK"}'
2025-12-26 04:13:25.050 DEBUG [tests.e2e.test_e2e] Check if service node1 receives message sent by edge node1
2025-12-26 04:13:25.051 DEBUG [src.steps.light_push] Lightpushing message
2025-12-26 04:13:25.052 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:57532/lightpush/v1/message" -H "Content-Type: application/json" -d '{"pubsubTopic": "/waku/2/rs/3/1", "message": {"payload": "RmlsdGVyIHdvcmtzISE=", "contentTopic": "/test/1/waku-filter/proto", "timestamp": 1766722405051562661}}'
2025-12-26 04:13:25.097 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-26 04:13:25.101 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2025-12-26 04:13:25.202 DEBUG [src.steps.light_push] Checking that peer NODE_1:wakuorg/nwaku:latest can find the lightpushed message
2025-12-26 04:13:25.202 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56969/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:13:25.205 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1766722405051562661,"ephemeral":false,"proof":""}]'
2025-12-26 04:13:25.207 DEBUG [tests.e2e.test_e2e] Check if edge node2 can query stored message
2025-12-26 04:13:25.207 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2025-12-26 04:13:25.208 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:26609/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F1&pageSize=50&ascending=true" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:13:25.235 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"4785f6d7de97b6e0994d","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x77a28bff68265ed58f8da02258e9d9108ebcff101b63bded1ff807f61ea9ad5e"}]}'
2025-12-26 04:13:25.236 DEBUG [src.steps.store] messages length is 1
2025-12-26 04:13:25.238 DEBUG [tests.e2e.test_e2e] Check if service node3 can query stored message
2025-12-26 04:13:25.238 DEBUG [src.steps.store] Checking that peer wakuorg/nwaku:latest can find the stored messages
2025-12-26 04:13:25.238 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:25145/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F1&pageSize=50&ascending=true" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:13:25.252 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"8efac6d364a9a5365522","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x77a28bff68265ed58f8da02258e9d9108ebcff101b63bded1ff807f61ea9ad5e"}]}'
2025-12-26 04:13:25.254 DEBUG [src.steps.store] messages length is 1
2025-12-26 04:13:25.256 DEBUG [tests.e2e.test_e2e] Check if edge node2 can get sent message using filter get request
2025-12-26 04:13:25.256 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:26609/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2025-12-26 04:13:25.259 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1766722405051562661,"ephemeral":false}]'
2025-12-26 04:13:25.261 DEBUG [tests.conftest] Running fixture teardown: test_setup
2025-12-26 04:13:25.262 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2025-12-26 04:13:25.262 DEBUG [src.node.waku_node] Stopping container with id 9a8f43011302
2025-12-26 04:13:25.825 DEBUG [src.node.waku_node] Container stopped.
2025-12-26 04:13:25.826 DEBUG [src.node.waku_node] Stopping container with id 404ef91c0234
2025-12-26 04:13:26.366 DEBUG [src.node.waku_node] Container stopped.
2025-12-26 04:13:26.367 DEBUG [src.node.waku_node] Stopping container with id b7d440516f3d
2025-12-26 04:13:26.728 ERROR [src.node.docker_mananger] Max retries reached for container 9a8f43011302. Exiting log stream.
2025-12-26 04:13:26.954 DEBUG [src.node.waku_node] Container stopped.
2025-12-26 04:13:26.954 DEBUG [src.node.waku_node] Stopping container with id aa33cc8eecbf
2025-12-26 04:13:27.180 ERROR [src.node.docker_mananger] Max retries reached for container 404ef91c0234. Exiting log stream.
2025-12-26 04:13:27.478 DEBUG [src.node.waku_node] Container stopped.
2025-12-26 04:13:27.479 DEBUG [src.node.waku_node] Stopping container with id 5245adb380aa
2025-12-26 04:13:27.767 ERROR [src.node.docker_mananger] Max retries reached for container b7d440516f3d. Exiting log stream.
2025-12-26 04:13:27.968 DEBUG [src.node.waku_node] Container stopped.
2025-12-26 04:13:27.968 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2025-12-26 04:13:28.022 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-26 04:13:28.027 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-26 04:13:28.054 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-26 04:13:28.078 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-26 04:13:28.085 DEBUG [src.node.docker_mananger] No errors found in the waku logs.