168 lines
32 KiB
Plaintext

2026-03-26 04:46:44.208 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-26 04:46:44.208 DEBUG [tests.conftest] Running test: test_all_nodes_subscribed_to_the_topic with id: 2026-03-26_04-46-44__db7df8ec-7a32-45f0-9f71-7f00f2f219e2
2026-03-26 04:46:44.209 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-26 04:46:44.209 DEBUG [src.steps.filter] Running fixture setup: filter_setup
2026-03-26 04:46:44.209 DEBUG [src.steps.filter] Running fixture setup: setup_main_relay_node
2026-03-26 04:46:44.216 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-26 04:46:44.217 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-26_04-46-44__db7df8ec-7a32-45f0-9f71-7f00f2f219e2__wakuorg_nwaku:latest.log
2026-03-26 04:46:44.217 DEBUG [src.node.waku_node] Starting Node...
2026-03-26 04:46:44.217 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-26 04:46:44.218 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-26 04:46:44.218 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.148.87
2026-03-26 04:46:44.219 DEBUG [src.node.docker_mananger] Generated ports ['47762', '47763', '47764', '47765', '47766']
2026-03-26 04:46:44.219 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-26 04:46:44.219 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-26 04:46:44.219 DEBUG [src.node.waku_node] Using volumes []
2026-03-26 04:46:44.219 DEBUG [src.node.docker_mananger] docker run -i -t -p 47762:47762 -p 47763:47763 -p 47764:47764 -p 47765:47765 -p 47766:47766 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=47764 --rest-port=47762 --tcp-port=47763 --discv5-udp-port=47765 --rest-address=0.0.0.0 --nat=extip:172.18.148.87 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=74ceddd0e2ff4aac86b08d4beabb480b0cffe8b5e6396b20dcb2f8afb298caaf --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=47766 --metrics-logging=true --relay=true --filter=true
2026-03-26 04:46:44.406 ERROR [src.node.docker_mananger] Max retries reached for container b71a8ed1a8ab. Exiting log stream.
2026-03-26 04:46:44.421 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.148.87 waku 06b48e1e1b178a77cbbb1f6483d1e6dc36aee14c284eb71441a280f3429e4265
2026-03-26 04:46:44.455 DEBUG [src.node.docker_mananger] Container started with ID 06b48e1e1b17. Setting up logs at ./log/docker/node1_2026-03-26_04-46-44__db7df8ec-7a32-45f0-9f71-7f00f2f219e2__wakuorg_nwaku:latest.log
2026-03-26 04:46:44.456 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 47762
2026-03-26 04:46:44.456 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-26 04:46:44.960 ERROR [src.node.docker_mananger] Max retries reached for container fe85887e85f4. Exiting log stream.
2026-03-26 04:46:45.456 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:47762/health" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:45.459 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-26 04:46:45.460 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-26 04:46:45.460 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:47762/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:45.462 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.148.87/tcp/47763/p2p/16Uiu2HAm2o5THqUH9gMcwERdPDR12sveBzC9KXxLRvAecxPKa7Li","/ip4/172.18.148.87/tcp/47764/ws/p2p/16Uiu2HAm2o5THqUH9gMcwERdPDR12sveBzC9KXxLRvAecxPKa7Li"],"enrUri":"enr:-L24QOFAWerpUh6Nhd7uJDhneJpDX7Cgzza6WC9ljo_YDtjGS_eWdPAUU0jTyjvBsjGBOYK2tgKt436lqFcLEl01pAECgmlkgnY0gmlwhKwSlFeKbXVsdGlhZGRyc5YACASsEpRXBrqTAAoErBKUVwa6lN0DgnJzhQADAQAAiXNlY3AyNTZrMaECbX2MN2GGS68VcExv-o5OvboBk8KrRDSykb9Hni13M5eDdGNwgrqTg3VkcIK6lYV3YWt1MgU"}'
2026-03-26 04:46:45.463 INFO [src.node.waku_node] REST service is ready !!
2026-03-26 04:46:45.463 DEBUG [src.steps.filter] Running fixture setup: setup_main_filter_node
2026-03-26 04:46:45.470 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-26 04:46:45.470 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-26_04-46-44__db7df8ec-7a32-45f0-9f71-7f00f2f219e2__wakuorg_nwaku:latest.log
2026-03-26 04:46:45.470 DEBUG [src.node.waku_node] Starting Node...
2026-03-26 04:46:45.470 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-26 04:46:45.471 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-26 04:46:45.472 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.98.57
2026-03-26 04:46:45.472 DEBUG [src.node.docker_mananger] Generated ports ['44937', '44938', '44939', '44940', '44941']
2026-03-26 04:46:45.472 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-26 04:46:45.472 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-26 04:46:45.472 DEBUG [src.node.waku_node] Using volumes []
2026-03-26 04:46:45.472 DEBUG [src.node.docker_mananger] docker run -i -t -p 44937:44937 -p 44938:44938 -p 44939:44939 -p 44940:44940 -p 44941:44941 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=44939 --rest-port=44937 --tcp-port=44938 --discv5-udp-port=44940 --rest-address=0.0.0.0 --nat=extip:172.18.98.57 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9aeacebe1dbdbab466bd38daef1f2555e21484ffa2677e0e810632e40e3682c5 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=44941 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QOFAWerpUh6Nhd7uJDhneJpDX7Cgzza6WC9ljo_YDtjGS_eWdPAUU0jTyjvBsjGBOYK2tgKt436lqFcLEl01pAECgmlkgnY0gmlwhKwSlFeKbXVsdGlhZGRyc5YACASsEpRXBrqTAAoErBKUVwa6lN0DgnJzhQADAQAAiXNlY3AyNTZrMaECbX2MN2GGS68VcExv-o5OvboBk8KrRDSykb9Hni13M5eDdGNwgrqTg3VkcIK6lYV3YWt1MgU --filternode=/ip4/172.18.148.87/tcp/47763/p2p/16Uiu2HAm2o5THqUH9gMcwERdPDR12sveBzC9KXxLRvAecxPKa7Li
2026-03-26 04:46:45.671 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.98.57 waku 3e08e1dc2461510ca5c059fb6291ff82c8107c442c62aaf9e27e5506507396b4
2026-03-26 04:46:45.705 DEBUG [src.node.docker_mananger] Container started with ID 3e08e1dc2461. Setting up logs at ./log/docker/node2_2026-03-26_04-46-44__db7df8ec-7a32-45f0-9f71-7f00f2f219e2__wakuorg_nwaku:latest.log
2026-03-26 04:46:45.706 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 44937
2026-03-26 04:46:45.706 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-26 04:46:46.707 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:44937/health" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:46.710 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-26 04:46:46.710 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-26 04:46:46.710 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:44937/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:46.712 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.98.57/tcp/44938/p2p/16Uiu2HAm2LwYGfrJTywCJMoE3UNQZJpfEWFFEjSK31KQzLxPiv8w","/ip4/172.18.98.57/tcp/44939/ws/p2p/16Uiu2HAm2LwYGfrJTywCJMoE3UNQZJpfEWFFEjSK31KQzLxPiv8w"],"enrUri":"enr:-L24QIpSGqz5elgyrEVIuu6KlDJyn73_CyfP1YMAPoFnZBvJDC2X3mcubAVn8Ony6Hn794LQgiKKw5lmqP8XwJC4iTUCgmlkgnY0gmlwhKwSYjmKbXVsdGlhZGRyc5YACASsEmI5Bq-KAAoErBJiOQavi90DgnJzhQADAQAAiXNlY3AyNTZrMaECZst_hfYTDOV56CMnRR0rPQ-nZFZoUMNIN2WbcZ93F6iDdGNwgq-Kg3VkcIKvjIV3YWt1MgA"}'
2026-03-26 04:46:46.713 INFO [src.node.waku_node] REST service is ready !!
2026-03-26 04:46:46.713 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:44937/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.148.87/tcp/47763/p2p/16Uiu2HAm2o5THqUH9gMcwERdPDR12sveBzC9KXxLRvAecxPKa7Li"]'
2026-03-26 04:46:46.744 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-26 04:46:46.756 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-26 04:46:46.756 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-26_04-46-44__db7df8ec-7a32-45f0-9f71-7f00f2f219e2__wakuorg_nwaku:latest.log
2026-03-26 04:46:46.757 DEBUG [src.node.waku_node] Starting Node...
2026-03-26 04:46:46.757 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-26 04:46:46.758 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-26 04:46:46.758 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.54.44
2026-03-26 04:46:46.758 DEBUG [src.node.docker_mananger] Generated ports ['36133', '36134', '36135', '36136', '36137']
2026-03-26 04:46:46.759 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-26 04:46:46.759 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-26 04:46:46.759 DEBUG [src.node.waku_node] Using volumes []
2026-03-26 04:46:46.759 DEBUG [src.node.docker_mananger] docker run -i -t -p 36133:36133 -p 36134:36134 -p 36135:36135 -p 36136:36136 -p 36137:36137 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=36135 --rest-port=36133 --tcp-port=36134 --discv5-udp-port=36136 --rest-address=0.0.0.0 --nat=extip:172.18.54.44 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=65a037b1da8ff17cae5f8aff72ce9cabb2a7cc283dcbfb5affea8d9ff8bce9bb --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=36137 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QOFAWerpUh6Nhd7uJDhneJpDX7Cgzza6WC9ljo_YDtjGS_eWdPAUU0jTyjvBsjGBOYK2tgKt436lqFcLEl01pAECgmlkgnY0gmlwhKwSlFeKbXVsdGlhZGRyc5YACASsEpRXBrqTAAoErBKUVwa6lN0DgnJzhQADAQAAiXNlY3AyNTZrMaECbX2MN2GGS68VcExv-o5OvboBk8KrRDSykb9Hni13M5eDdGNwgrqTg3VkcIK6lYV3YWt1MgU --filternode=/ip4/172.18.148.87/tcp/47763/p2p/16Uiu2HAm2o5THqUH9gMcwERdPDR12sveBzC9KXxLRvAecxPKa7Li
2026-03-26 04:46:46.962 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.54.44 waku 1714c4c0145c00eb53d8eb90e55e416e031fa15adcaa89d022863215f10c1511
2026-03-26 04:46:47.001 DEBUG [src.node.docker_mananger] Container started with ID 1714c4c0145c. Setting up logs at ./log/docker/node3_2026-03-26_04-46-44__db7df8ec-7a32-45f0-9f71-7f00f2f219e2__wakuorg_nwaku:latest.log
2026-03-26 04:46:47.002 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 36133
2026-03-26 04:46:47.002 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-26 04:46:48.004 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36133/health" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:48.007 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-26 04:46:48.007 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-26 04:46:48.007 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36133/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:48.010 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.54.44/tcp/36134/p2p/16Uiu2HAkwaaScoDAn2R2ayfoVxQKxa1pPwTjA5jZGt6YrGabZE6E","/ip4/172.18.54.44/tcp/36135/ws/p2p/16Uiu2HAkwaaScoDAn2R2ayfoVxQKxa1pPwTjA5jZGt6YrGabZE6E"],"enrUri":"enr:-L24QKc2GeuJQqN21A6GB3squPjXRQFiECExNx_jQ7kGu8BZI4gfFzfr3Bymq4BKBZ6vTpxuqoI0r27gFhkDHJ5AEAsCgmlkgnY0gmlwhKwSNiyKbXVsdGlhZGRyc5YACASsEjYsBo0mAAoErBI2LAaNJ90DgnJzhQADAQAAiXNlY3AyNTZrMaECH_9XAJ0Yihbp0ogbOTcW-lfvFsxtSm7N6p41lX5eXAODdGNwgo0mg3VkcIKNKIV3YWt1MgA"}'
2026-03-26 04:46:48.010 INFO [src.node.waku_node] REST service is ready !!
2026-03-26 04:46:48.010 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:36133/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.148.87/tcp/47763/p2p/16Uiu2HAm2o5THqUH9gMcwERdPDR12sveBzC9KXxLRvAecxPKa7Li"]'
2026-03-26 04:46:48.042 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-26 04:46:48.053 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-26 04:46:48.054 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-26_04-46-44__db7df8ec-7a32-45f0-9f71-7f00f2f219e2__wakuorg_nwaku:latest.log
2026-03-26 04:46:48.055 DEBUG [src.node.waku_node] Starting Node...
2026-03-26 04:46:48.055 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-26 04:46:48.058 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-26 04:46:48.058 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.152.215
2026-03-26 04:46:48.058 DEBUG [src.node.docker_mananger] Generated ports ['33256', '33257', '33258', '33259', '33260']
2026-03-26 04:46:48.058 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-26 04:46:48.058 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-26 04:46:48.058 DEBUG [src.node.waku_node] Using volumes []
2026-03-26 04:46:48.058 DEBUG [src.node.docker_mananger] docker run -i -t -p 33256:33256 -p 33257:33257 -p 33258:33258 -p 33259:33259 -p 33260:33260 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=33258 --rest-port=33256 --tcp-port=33257 --discv5-udp-port=33259 --rest-address=0.0.0.0 --nat=extip:172.18.152.215 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9696b5c5fe0626aee4caddaaafd6fdaddce87df5e44e75d5a5cea343e1cb0f97 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=33260 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QOFAWerpUh6Nhd7uJDhneJpDX7Cgzza6WC9ljo_YDtjGS_eWdPAUU0jTyjvBsjGBOYK2tgKt436lqFcLEl01pAECgmlkgnY0gmlwhKwSlFeKbXVsdGlhZGRyc5YACASsEpRXBrqTAAoErBKUVwa6lN0DgnJzhQADAQAAiXNlY3AyNTZrMaECbX2MN2GGS68VcExv-o5OvboBk8KrRDSykb9Hni13M5eDdGNwgrqTg3VkcIK6lYV3YWt1MgU --filternode=/ip4/172.18.148.87/tcp/47763/p2p/16Uiu2HAm2o5THqUH9gMcwERdPDR12sveBzC9KXxLRvAecxPKa7Li
2026-03-26 04:46:48.259 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.152.215 waku bbe841166dbea5b1329d89cf97d76c63b3ca03df517883c3a90d049464fd4ee8
2026-03-26 04:46:48.293 DEBUG [src.node.docker_mananger] Container started with ID bbe841166dbe. Setting up logs at ./log/docker/node4_2026-03-26_04-46-44__db7df8ec-7a32-45f0-9f71-7f00f2f219e2__wakuorg_nwaku:latest.log
2026-03-26 04:46:48.294 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 33256
2026-03-26 04:46:48.294 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-26 04:46:49.294 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33256/health" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:49.298 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-26 04:46:49.298 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-26 04:46:49.298 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33256/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:49.300 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.152.215/tcp/33257/p2p/16Uiu2HAm83Hgdfz1kPJdsBLpbjSkCSGxV2q4BoaGLZSY5aSp16mZ","/ip4/172.18.152.215/tcp/33258/ws/p2p/16Uiu2HAm83Hgdfz1kPJdsBLpbjSkCSGxV2q4BoaGLZSY5aSp16mZ"],"enrUri":"enr:-L24QCXNy6LpZxDOqRiND_qUstlh6OqvAF12xmMjTLAI2uB9JopS5x-cstl6FLmctxKKxTpLchcNmvb48pCXNsEXvIwCgmlkgnY0gmlwhKwSmNeKbXVsdGlhZGRyc5YACASsEpjXBoHpAAoErBKY1waB6t0DgnJzhQADAQAAiXNlY3AyNTZrMaECu2vxNYgXRq3eOZbtwlzKrlmENNPZg8SqI2yUV3zC15yDdGNwgoHpg3VkcIKB64V3YWt1MgA"}'
2026-03-26 04:46:49.300 INFO [src.node.waku_node] REST service is ready !!
2026-03-26 04:46:49.301 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:33256/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.148.87/tcp/47763/p2p/16Uiu2HAm2o5THqUH9gMcwERdPDR12sveBzC9KXxLRvAecxPKa7Li"]'
2026-03-26 04:46:49.333 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-26 04:46:49.348 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-26 04:46:49.348 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-03-26_04-46-44__db7df8ec-7a32-45f0-9f71-7f00f2f219e2__wakuorg_nwaku:latest.log
2026-03-26 04:46:49.348 DEBUG [src.node.waku_node] Starting Node...
2026-03-26 04:46:49.348 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-26 04:46:49.349 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-26 04:46:49.350 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.233.101
2026-03-26 04:46:49.350 DEBUG [src.node.docker_mananger] Generated ports ['35474', '35475', '35476', '35477', '35478']
2026-03-26 04:46:49.350 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-26 04:46:49.350 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-26 04:46:49.350 DEBUG [src.node.waku_node] Using volumes []
2026-03-26 04:46:49.350 DEBUG [src.node.docker_mananger] docker run -i -t -p 35474:35474 -p 35475:35475 -p 35476:35476 -p 35477:35477 -p 35478:35478 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=35476 --rest-port=35474 --tcp-port=35475 --discv5-udp-port=35477 --rest-address=0.0.0.0 --nat=extip:172.18.233.101 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=b748aaec8ef0feb11accd33261bb93afd1365e16c03b5b5fdbbf0394bdeb4dde --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=35478 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QOFAWerpUh6Nhd7uJDhneJpDX7Cgzza6WC9ljo_YDtjGS_eWdPAUU0jTyjvBsjGBOYK2tgKt436lqFcLEl01pAECgmlkgnY0gmlwhKwSlFeKbXVsdGlhZGRyc5YACASsEpRXBrqTAAoErBKUVwa6lN0DgnJzhQADAQAAiXNlY3AyNTZrMaECbX2MN2GGS68VcExv-o5OvboBk8KrRDSykb9Hni13M5eDdGNwgrqTg3VkcIK6lYV3YWt1MgU --filternode=/ip4/172.18.148.87/tcp/47763/p2p/16Uiu2HAm2o5THqUH9gMcwERdPDR12sveBzC9KXxLRvAecxPKa7Li
2026-03-26 04:46:49.555 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.233.101 waku 2169fef840a425fa2d42d11f2fb7ae5a261518620ed2322f978dc60c0d11dd1e
2026-03-26 04:46:49.588 DEBUG [src.node.docker_mananger] Container started with ID 2169fef840a4. Setting up logs at ./log/docker/node5_2026-03-26_04-46-44__db7df8ec-7a32-45f0-9f71-7f00f2f219e2__wakuorg_nwaku:latest.log
2026-03-26 04:46:49.589 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 35474
2026-03-26 04:46:49.590 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-26 04:46:50.590 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35474/health" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:50.594 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-26 04:46:50.594 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-26 04:46:50.594 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35474/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:50.596 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.233.101/tcp/35475/p2p/16Uiu2HAm8XPyeiWF3ivj9bfiAEd6SHK9JqYtTGo7443kbfFAyQQi","/ip4/172.18.233.101/tcp/35476/ws/p2p/16Uiu2HAm8XPyeiWF3ivj9bfiAEd6SHK9JqYtTGo7443kbfFAyQQi"],"enrUri":"enr:-L24QIhDxCVKuhNu1cFH97HYVPlr2ZS0Y_e4e0VRbsp8OR5TZ_kh4GwXA7mZVT1nkoD-JwEArkIkaCmzKNISDLac-tkCgmlkgnY0gmlwhKwS6WWKbXVsdGlhZGRyc5YACASsEullBoqTAAoErBLpZQaKlN0DgnJzhQADAQAAiXNlY3AyNTZrMaECwp9SJA4wMfT8jOwYgEHHC1_fywRGzCJK_4rGlDsehiuDdGNwgoqTg3VkcIKKlYV3YWt1MgA"}'
2026-03-26 04:46:50.596 INFO [src.node.waku_node] REST service is ready !!
2026-03-26 04:46:50.597 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35474/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.148.87/tcp/47763/p2p/16Uiu2HAm2o5THqUH9gMcwERdPDR12sveBzC9KXxLRvAecxPKa7Li"]'
2026-03-26 04:46:50.627 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-26 04:46:50.632 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:47762/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-26 04:46:50.639 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-26 04:46:50.640 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:44937/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "2ce399f0-3619-4d32-a36f-16f47c3179b2", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-26 04:46:50.653 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"2ce399f0-3619-4d32-a36f-16f47c3179b2","statusDesc":"OK"}'
2026-03-26 04:46:50.654 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:36133/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "10b3866a-f770-41ca-9f13-701bd3a09acb", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-26 04:46:50.664 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"10b3866a-f770-41ca-9f13-701bd3a09acb","statusDesc":"OK"}'
2026-03-26 04:46:50.664 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:33256/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "feab314f-8d98-49d8-bb1b-12be80363889", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-26 04:46:50.673 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"feab314f-8d98-49d8-bb1b-12be80363889","statusDesc":"OK"}'
2026-03-26 04:46:50.673 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35474/filter/v2/subscriptions" -H "Content-Type: application/json" -d '{"requestId": "2d800af3-b417-499b-87df-4daa5b1c6ea9", "contentFilters": ["/test/1/waku-filter/proto"], "pubsubTopic": "/waku/2/rs/3/1"}'
2026-03-26 04:46:50.682 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"2d800af3-b417-499b-87df-4daa5b1c6ea9","statusDesc":"OK"}'
2026-03-26 04:46:50.683 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:47762/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "RmlsdGVyIHdvcmtzISE=", "contentTopic": "/test/1/waku-filter/proto", "timestamp": '$(date +%s%N)'}'
2026-03-26 04:46:50.698 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-26 04:46:50.699 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-26 04:46:50.799 DEBUG [src.steps.filter] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-26 04:46:50.799 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:44937/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:50.802 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1774500410683173530,"ephemeral":false}]'
2026-03-26 04:46:50.804 DEBUG [src.steps.filter] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-03-26 04:46:50.804 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36133/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:50.806 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1774500410683173530,"ephemeral":false}]'
2026-03-26 04:46:50.807 DEBUG [src.steps.filter] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-03-26 04:46:50.808 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33256/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:50.810 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1774500410683173530,"ephemeral":false}]'
2026-03-26 04:46:50.811 DEBUG [src.steps.filter] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-03-26 04:46:50.812 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35474/filter/v2/messages/%2Ftest%2F1%2Fwaku-filter%2Fproto" -H "Content-Type: application/json" -d 'None'
2026-03-26 04:46:50.814 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"RmlsdGVyIHdvcmtzISE=","contentTopic":"/test/1/waku-filter/proto","version":0,"timestamp":1774500410683173530,"ephemeral":false}]'
2026-03-26 04:46:50.817 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-26 04:46:50.818 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-26 04:46:50.818 DEBUG [src.node.waku_node] Stopping container with id 06b48e1e1b17
2026-03-26 04:46:51.410 DEBUG [src.node.waku_node] Container stopped.
2026-03-26 04:46:51.412 DEBUG [src.node.waku_node] Stopping container with id 3e08e1dc2461
2026-03-26 04:46:51.987 DEBUG [src.node.waku_node] Container stopped.
2026-03-26 04:46:51.988 DEBUG [src.node.waku_node] Stopping container with id 1714c4c0145c
2026-03-26 04:46:52.224 ERROR [src.node.docker_mananger] Max retries reached for container 06b48e1e1b17. Exiting log stream.
2026-03-26 04:46:52.595 DEBUG [src.node.waku_node] Container stopped.
2026-03-26 04:46:52.598 DEBUG [src.node.waku_node] Stopping container with id bbe841166dbe
2026-03-26 04:46:52.819 ERROR [src.node.docker_mananger] Max retries reached for container 3e08e1dc2461. Exiting log stream.
2026-03-26 04:46:53.164 DEBUG [src.node.waku_node] Container stopped.
2026-03-26 04:46:53.166 DEBUG [src.node.waku_node] Stopping container with id 2169fef840a4
2026-03-26 04:46:53.410 ERROR [src.node.docker_mananger] Max retries reached for container 1714c4c0145c. Exiting log stream.
2026-03-26 04:46:53.759 DEBUG [src.node.waku_node] Container stopped.
2026-03-26 04:46:53.760 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-26 04:46:53.777 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-26 04:46:53.787 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-26 04:46:53.792 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-26 04:46:53.797 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-26 04:46:53.801 DEBUG [src.node.docker_mananger] No errors found in the waku logs.