201 lines
39 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

INFO  tests.conftest:conftest.py:106 Fleet bootstrap inactive pass --fleet (or set FLEET_BOOTSTRAP=true) to connect local nodes to the waku.test fleet
DEBUG  tests.conftest:conftest.py:201 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:207 Running test: test_multiple_content_topics_different_shard with id: 2026-04-23_00-05-03__1b0fee28-eae6-4d28-afb2-1148ce2f9af4
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:108 WakuNode instance initialized with log path ./log/docker/node1_2026-04-23_00-05-03__1b0fee28-eae6-4d28-afb2-1148ce2f9af4__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:116 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.202.2
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['58142', '58143', '58144', '58145', '58146']
DEBUG  src.node.waku_node:waku_node.py:606 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:211 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:217 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 58142:58142 -p 58143:58143 -p 58144:58144 -p 58145:58145 -p 58146:58146 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=58144 --rest-port=58142 --tcp-port=58143 --discv5-udp-port=58145 --rest-address=0.0.0.0 --nat=extip:172.18.202.2 --peer-exchange=true --discv5-discovery=true --cluster-id=199 --nodekey=8defe3cb03ebb9482a1b618ef658f94d20bddfd797a3c2fbdccc5ba4e8f1f6bb --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=58146 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto --content-topic=/waku/2/content/test.js --content-topic=/app/22/sometopic/someencoding --content-topic=/toychat/2/huilong/proto --content-topic=/statusim/1/community/cbor --content-topic=/app/27/sometopic/someencoding --content-topic=/app/29/sometopic/someencoding --content-topic=/app/20/sometopic/someencoding
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.202.2 waku 033f701967e61b8f8fa0c8588769f520e3325acfa61d756f7894f8f9546f6cc3
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 033f701967e6. Setting up logs at ./log/docker/node1_2026-04-23_00-05-03__1b0fee28-eae6-4d28-afb2-1148ce2f9af4__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:228 Started container from image wakuorg/nwaku:latest. REST: 58142
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 467eecbf12bd. Exiting log stream.
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 6dc0bd41c7ef. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:58142/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:412 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:58142/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.202.2/tcp/58143/p2p/16Uiu2HAkyUXpvSDZSNsu28pYzr8ywqYRGF3PXTkMjzFVHzk6wUjd","/ip4/172.18.202.2/tcp/58144/ws/p2p/16Uiu2HAkyUXpvSDZSNsu28pYzr8ywqYRGF3PXTkMjzFVHzk6wUjd"],"enrUri":"enr:-L24QKA5tVdiW0S3HmOE23kiBJg_rytgyws8h1-xfC9VQeAVdh0BFHMj_imlhivjwfawKXh1Mfh1WlnrJc_WSnD0P_ICgmlkgnY0gmlwhKwSygKKbXVsdGlhZGRyc5YACASsEsoCBuMfAAoErBLKAgbjIN0DgnJzhQDHAQAAiXNlY3AyNTZrMaECPCpFhLva4BojIfrklNTPx8sOse_Fl2pWCln9SsFjvFSDdGNwguMfg3VkcILjIYV3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:417 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:108 WakuNode instance initialized with log path ./log/docker/node2_2026-04-23_00-05-03__1b0fee28-eae6-4d28-afb2-1148ce2f9af4__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:116 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.145.74
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['50056', '50057', '50058', '50059', '50060']
DEBUG  src.node.waku_node:waku_node.py:606 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:211 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:217 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 50056:50056 -p 50057:50057 -p 50058:50058 -p 50059:50059 -p 50060:50060 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=50058 --rest-port=50056 --tcp-port=50057 --discv5-udp-port=50059 --rest-address=0.0.0.0 --nat=extip:172.18.145.74 --peer-exchange=true --discv5-discovery=true --cluster-id=199 --nodekey=b4e8bb0bdcce8cbddfbb8cd4a2e5dcd2f75ac51a4fe8ffea3f1e5c3bd520fa29 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=50060 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QKA5tVdiW0S3HmOE23kiBJg_rytgyws8h1-xfC9VQeAVdh0BFHMj_imlhivjwfawKXh1Mfh1WlnrJc_WSnD0P_ICgmlkgnY0gmlwhKwSygKKbXVsdGlhZGRyc5YACASsEsoCBuMfAAoErBLKAgbjIN0DgnJzhQDHAQAAiXNlY3AyNTZrMaECPCpFhLva4BojIfrklNTPx8sOse_Fl2pWCln9SsFjvFSDdGNwguMfg3VkcILjIYV3YWt1MgU --content-topic=/myapp/1/latest/proto --content-topic=/waku/2/content/test.js --content-topic=/app/22/sometopic/someencoding --content-topic=/toychat/2/huilong/proto --content-topic=/statusim/1/community/cbor --content-topic=/app/27/sometopic/someencoding --content-topic=/app/29/sometopic/someencoding --content-topic=/app/20/sometopic/someencoding
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.145.74 waku b0d120aa28381e0030737093a224490493bff7ff593ddf562fb6779aa4341bc8
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID b0d120aa2838. Setting up logs at ./log/docker/node2_2026-04-23_00-05-03__1b0fee28-eae6-4d28-afb2-1148ce2f9af4__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:228 Started container from image wakuorg/nwaku:latest. REST: 50056
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:50056/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:412 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:50056/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.145.74/tcp/50057/p2p/16Uiu2HAm5dhW9VamTucS3pAJMsGp2hpm55rTfP4hhqyKKMb6X7TP","/ip4/172.18.145.74/tcp/50058/ws/p2p/16Uiu2HAm5dhW9VamTucS3pAJMsGp2hpm55rTfP4hhqyKKMb6X7TP"],"enrUri":"enr:-L24QBZ8BBKMxTjIclB934nIx_xnV_H7DnvO9VIh0BeB8mdZW87P14g-ztvKw5NFtBRO2WNG5xc9TnglShAdwspNVPUCgmlkgnY0gmlwhKwSkUqKbXVsdGlhZGRyc5YACASsEpFKBsOJAAoErBKRSgbDit0DgnJzhQDHAQAAiXNlY3AyNTZrMaECl6mPuSDqYtE6tDr1OwfcL79VzJXZLOLuDKsslUZwAhKDdGNwgsOJg3VkcILDi4V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:417 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:50056/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.202.2/tcp/58143/p2p/16Uiu2HAkyUXpvSDZSNsu28pYzr8ywqYRGF3PXTkMjzFVHzk6wUjd"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:58142/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
ERROR  src.node.api_clients.base_client:base_client.py:16 HTTP error occurred: 500 Server Error: Internal Server Error for url: http://127.0.0.1:58142/relay/v1/auto/subscriptions. Response content: b'Subscribe failed:Failed to decode subscription event: Static sharding is used, relay subscriptions must specify a pubsub topic'
DEBUG  tests.conftest:conftest.py:209 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:233 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:343 Stopping container with id 033f701967e6
DEBUG  src.node.waku_node:waku_node.py:355 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:343 Stopping container with id b0d120aa2838
DEBUG  src.node.waku_node:waku_node.py:355 Container stopped.
DEBUG  tests.conftest:conftest.py:248 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:223 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:224 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-04-23_00-05-03__1b0fee28-eae6-4d28-afb2-1148ce2f9af4__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-04-23_00-05-03__1b0fee28-eae6-4d28-afb2-1148ce2f9af4__wakuorg_nwaku:latest.log
INFO  tests.conftest:conftest.py:106 Fleet bootstrap inactive pass --fleet (or set FLEET_BOOTSTRAP=true) to connect local nodes to the waku.test fleet
DEBUG  tests.conftest:conftest.py:201 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:207 Running test: test_multiple_content_topics_different_shard with id: 2026-04-23_00-05-06__d55306f1-03c8-4abe-8d57-7f703d77b658
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:108 WakuNode instance initialized with log path ./log/docker/node1_2026-04-23_00-05-06__d55306f1-03c8-4abe-8d57-7f703d77b658__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:116 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.149.109
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['61211', '61212', '61213', '61214', '61215']
DEBUG  src.node.waku_node:waku_node.py:606 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:211 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:217 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 61211:61211 -p 61212:61212 -p 61213:61213 -p 61214:61214 -p 61215:61215 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=61213 --rest-port=61211 --tcp-port=61212 --discv5-udp-port=61214 --rest-address=0.0.0.0 --nat=extip:172.18.149.109 --peer-exchange=true --discv5-discovery=true --cluster-id=199 --nodekey=6e0c371d9dee29a6c9b496f663a82cffb7c3af217c7fb6aeaaaec574fc7a9ff6 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=61215 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto --content-topic=/waku/2/content/test.js --content-topic=/app/22/sometopic/someencoding --content-topic=/toychat/2/huilong/proto --content-topic=/statusim/1/community/cbor --content-topic=/app/27/sometopic/someencoding --content-topic=/app/29/sometopic/someencoding --content-topic=/app/20/sometopic/someencoding
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.149.109 waku bf1cbc40df3fc876f3ebbf231170f40c593a3e917010718338337cd4e25c733c
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID bf1cbc40df3f. Setting up logs at ./log/docker/node1_2026-04-23_00-05-06__d55306f1-03c8-4abe-8d57-7f703d77b658__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:228 Started container from image wakuorg/nwaku:latest. REST: 61211
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 033f701967e6. Exiting log stream.
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container b0d120aa2838. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:61211/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:412 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:61211/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.149.109/tcp/61212/p2p/16Uiu2HAmK7P6CEfahbCYfe36DNmaBeq6fPskSjwTH5yuqiE3YktP","/ip4/172.18.149.109/tcp/61213/ws/p2p/16Uiu2HAmK7P6CEfahbCYfe36DNmaBeq6fPskSjwTH5yuqiE3YktP"],"enrUri":"enr:-L24QNaFAv8V7bDB3WYJUVKa_MAm-NCbErEjAZzyjrUZ_4cML3YzNpGhdxPPGJwih9U-lTDkRkOtmtfqpBBWoR8_UxoCgmlkgnY0gmlwhKwSlW2KbXVsdGlhZGRyc5YACASsEpVtBu8cAAoErBKVbQbvHd0DgnJzhQDHAQAAiXNlY3AyNTZrMaEDX-jvOw959IPj9Q8jgOFuFjB0j47-rlevLAyF3GMtiwiDdGNwgu8cg3VkcILvHoV3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:417 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:108 WakuNode instance initialized with log path ./log/docker/node2_2026-04-23_00-05-06__d55306f1-03c8-4abe-8d57-7f703d77b658__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:116 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.109.171
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['6290', '6291', '6292', '6293', '6294']
DEBUG  src.node.waku_node:waku_node.py:606 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:211 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:217 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 6290:6290 -p 6291:6291 -p 6292:6292 -p 6293:6293 -p 6294:6294 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=6292 --rest-port=6290 --tcp-port=6291 --discv5-udp-port=6293 --rest-address=0.0.0.0 --nat=extip:172.18.109.171 --peer-exchange=true --discv5-discovery=true --cluster-id=199 --nodekey=2fe7badac95a4926dfeb16daceab6c123ab310aae5425c8d20c0ef39aa52a13f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=6294 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNaFAv8V7bDB3WYJUVKa_MAm-NCbErEjAZzyjrUZ_4cML3YzNpGhdxPPGJwih9U-lTDkRkOtmtfqpBBWoR8_UxoCgmlkgnY0gmlwhKwSlW2KbXVsdGlhZGRyc5YACASsEpVtBu8cAAoErBKVbQbvHd0DgnJzhQDHAQAAiXNlY3AyNTZrMaEDX-jvOw959IPj9Q8jgOFuFjB0j47-rlevLAyF3GMtiwiDdGNwgu8cg3VkcILvHoV3YWt1MgU --content-topic=/myapp/1/latest/proto --content-topic=/waku/2/content/test.js --content-topic=/app/22/sometopic/someencoding --content-topic=/toychat/2/huilong/proto --content-topic=/statusim/1/community/cbor --content-topic=/app/27/sometopic/someencoding --content-topic=/app/29/sometopic/someencoding --content-topic=/app/20/sometopic/someencoding
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.109.171 waku 19e7dd5dffb6fe83b46fecf4e8ceb3c8b8a7536fda0c99cdda42f429fe26dd59
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 19e7dd5dffb6. Setting up logs at ./log/docker/node2_2026-04-23_00-05-06__d55306f1-03c8-4abe-8d57-7f703d77b658__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:228 Started container from image wakuorg/nwaku:latest. REST: 6290
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:6290/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:412 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:6290/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.109.171/tcp/6291/p2p/16Uiu2HAkxxXGbPwEGfE4mBLFMVvwFdbtVW3djBqxR5uvyJDa3Mpc","/ip4/172.18.109.171/tcp/6292/ws/p2p/16Uiu2HAkxxXGbPwEGfE4mBLFMVvwFdbtVW3djBqxR5uvyJDa3Mpc"],"enrUri":"enr:-L24QFX4WAAoT5gaeNa3gymdtWRnlqaGB3WTRX14Sr8f07zaVVRJ88cR7cdkQyriH7uKmg_idVwd_NQ0QOSp_Wt88XkCgmlkgnY0gmlwhKwSbauKbXVsdGlhZGRyc5YACASsEm2rBhiTAAoErBJtqwYYlN0DgnJzhQDHAQAAiXNlY3AyNTZrMaECNHo3kHv63vIYgt_OjlhS6ZF9dR_5tvhFDYDGcihKC7mDdGNwghiTg3VkcIIYlYV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:417 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:6290/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.149.109/tcp/61212/p2p/16Uiu2HAmK7P6CEfahbCYfe36DNmaBeq6fPskSjwTH5yuqiE3YktP"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:61211/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
ERROR  src.node.api_clients.base_client:base_client.py:16 HTTP error occurred: 500 Server Error: Internal Server Error for url: http://127.0.0.1:61211/relay/v1/auto/subscriptions. Response content: b'Subscribe failed:Failed to decode subscription event: Static sharding is used, relay subscriptions must specify a pubsub topic'
DEBUG  tests.conftest:conftest.py:209 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:233 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:343 Stopping container with id bf1cbc40df3f
DEBUG  src.node.waku_node:waku_node.py:355 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:343 Stopping container with id 19e7dd5dffb6
DEBUG  src.node.waku_node:waku_node.py:355 Container stopped.
DEBUG  tests.conftest:conftest.py:248 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:223 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:224 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-04-23_00-05-06__d55306f1-03c8-4abe-8d57-7f703d77b658__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-04-23_00-05-06__d55306f1-03c8-4abe-8d57-7f703d77b658__wakuorg_nwaku:latest.log
INFO  tests.conftest:conftest.py:106 Fleet bootstrap inactive pass --fleet (or set FLEET_BOOTSTRAP=true) to connect local nodes to the waku.test fleet
DEBUG  tests.conftest:conftest.py:201 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:207 Running test: test_multiple_content_topics_different_shard with id: 2026-04-23_00-05-10__97ef5ee1-a7e6-4ba7-b758-573d7051f1dc
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:108 WakuNode instance initialized with log path ./log/docker/node1_2026-04-23_00-05-10__97ef5ee1-a7e6-4ba7-b758-573d7051f1dc__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:116 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.102.142
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['33711', '33712', '33713', '33714', '33715']
DEBUG  src.node.waku_node:waku_node.py:606 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:211 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:217 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 33711:33711 -p 33712:33712 -p 33713:33713 -p 33714:33714 -p 33715:33715 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=33713 --rest-port=33711 --tcp-port=33712 --discv5-udp-port=33714 --rest-address=0.0.0.0 --nat=extip:172.18.102.142 --peer-exchange=true --discv5-discovery=true --cluster-id=199 --nodekey=cc9aeef9a5aeee3a029b5cdfe51abdd81db8418342a5b24e8a7abd3d1af20cd7 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=33715 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto --content-topic=/waku/2/content/test.js --content-topic=/app/22/sometopic/someencoding --content-topic=/toychat/2/huilong/proto --content-topic=/statusim/1/community/cbor --content-topic=/app/27/sometopic/someencoding --content-topic=/app/29/sometopic/someencoding --content-topic=/app/20/sometopic/someencoding
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.102.142 waku 103cd5f117b1aae71800f77df179c40df3367c3f054414abd08c685ca8a67349
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 103cd5f117b1. Setting up logs at ./log/docker/node1_2026-04-23_00-05-10__97ef5ee1-a7e6-4ba7-b758-573d7051f1dc__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:228 Started container from image wakuorg/nwaku:latest. REST: 33711
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container bf1cbc40df3f. Exiting log stream.
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 19e7dd5dffb6. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:33711/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:412 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:33711/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.102.142/tcp/33712/p2p/16Uiu2HAmVbJd18aa1hj9Dn4W3PWjbzB1cVuoFctqveSaTM8GDJNT","/ip4/172.18.102.142/tcp/33713/ws/p2p/16Uiu2HAmVbJd18aa1hj9Dn4W3PWjbzB1cVuoFctqveSaTM8GDJNT"],"enrUri":"enr:-L24QPvAY9gUZM1M6AM4f830U5DkF_JPjtZDRR4u0e9I-VKoJfV1SFCwA-Bhd8gwSJbytgVTYlqkhI6fkaX-DqTWWdoCgmlkgnY0gmlwhKwSZo6KbXVsdGlhZGRyc5YACASsEmaOBoOwAAoErBJmjgaDsd0DgnJzhQDHAQAAiXNlY3AyNTZrMaED-6UDdPfnNEhTCJy0fZNnw6FCfshuQ5oGtsR_unSXznCDdGNwgoOwg3VkcIKDsoV3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:417 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:108 WakuNode instance initialized with log path ./log/docker/node2_2026-04-23_00-05-10__97ef5ee1-a7e6-4ba7-b758-573d7051f1dc__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:116 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.39.229
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['23738', '23739', '23740', '23741', '23742']
DEBUG  src.node.waku_node:waku_node.py:606 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:211 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:217 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 23738:23738 -p 23739:23739 -p 23740:23740 -p 23741:23741 -p 23742:23742 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=23740 --rest-port=23738 --tcp-port=23739 --discv5-udp-port=23741 --rest-address=0.0.0.0 --nat=extip:172.18.39.229 --peer-exchange=true --discv5-discovery=true --cluster-id=199 --nodekey=e7b6cef631d4065a38a389be6a651e757587dec6b6c3a7f68aeba682ee5ce0ac --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=23742 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPvAY9gUZM1M6AM4f830U5DkF_JPjtZDRR4u0e9I-VKoJfV1SFCwA-Bhd8gwSJbytgVTYlqkhI6fkaX-DqTWWdoCgmlkgnY0gmlwhKwSZo6KbXVsdGlhZGRyc5YACASsEmaOBoOwAAoErBJmjgaDsd0DgnJzhQDHAQAAiXNlY3AyNTZrMaED-6UDdPfnNEhTCJy0fZNnw6FCfshuQ5oGtsR_unSXznCDdGNwgoOwg3VkcIKDsoV3YWt1MgU --content-topic=/myapp/1/latest/proto --content-topic=/waku/2/content/test.js --content-topic=/app/22/sometopic/someencoding --content-topic=/toychat/2/huilong/proto --content-topic=/statusim/1/community/cbor --content-topic=/app/27/sometopic/someencoding --content-topic=/app/29/sometopic/someencoding --content-topic=/app/20/sometopic/someencoding
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.39.229 waku a07f0ffd95cfd2647ae35af634e5090899b85eea32a14f597b56a6c5cd4b23ca
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID a07f0ffd95cf. Setting up logs at ./log/docker/node2_2026-04-23_00-05-10__97ef5ee1-a7e6-4ba7-b758-573d7051f1dc__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:228 Started container from image wakuorg/nwaku:latest. REST: 23738
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:23738/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:412 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:23738/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.39.229/tcp/23739/p2p/16Uiu2HAkyzuCdLju9S7GLxrsqTcNW2R1MDmuLSuuKkfmXBG7FcEK","/ip4/172.18.39.229/tcp/23740/ws/p2p/16Uiu2HAkyzuCdLju9S7GLxrsqTcNW2R1MDmuLSuuKkfmXBG7FcEK"],"enrUri":"enr:-L24QLwkw-La8CBTWt-o00cctPyxfJlm-vlMlEpSg6HoJG3TFosgPlF50t71KN6IMPbyEINga3IQiMmJUcOk6NhZHVQCgmlkgnY0gmlwhKwSJ-WKbXVsdGlhZGRyc5YACASsEiflBly7AAoErBIn5QZcvN0DgnJzhQDHAQAAiXNlY3AyNTZrMaECQ_HdHnt5h49qZ3Wj1UrfOBO03ogcm-LV-DWm9_RrzWCDdGNwgly7g3VkcIJcvYV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:417 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:23738/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.102.142/tcp/33712/p2p/16Uiu2HAmVbJd18aa1hj9Dn4W3PWjbzB1cVuoFctqveSaTM8GDJNT"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:33711/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto", "/waku/2/content/test.js", "/app/22/sometopic/someencoding", "/toychat/2/huilong/proto", "/statusim/1/community/cbor", "/app/27/sometopic/someencoding", "/app/29/sometopic/someencoding", "/app/20/sometopic/someencoding"]'
ERROR  src.node.api_clients.base_client:base_client.py:16 HTTP error occurred: 500 Server Error: Internal Server Error for url: http://127.0.0.1:33711/relay/v1/auto/subscriptions. Response content: b'Subscribe failed:Failed to decode subscription event: Static sharding is used, relay subscriptions must specify a pubsub topic'
DEBUG  tests.conftest:conftest.py:209 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:233 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:343 Stopping container with id 103cd5f117b1
DEBUG  src.node.waku_node:waku_node.py:355 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:343 Stopping container with id a07f0ffd95cf
DEBUG  src.node.waku_node:waku_node.py:355 Container stopped.
DEBUG  tests.conftest:conftest.py:248 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:223 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:224 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-04-23_00-05-10__97ef5ee1-a7e6-4ba7-b758-573d7051f1dc__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-04-23_00-05-10__97ef5ee1-a7e6-4ba7-b758-573d7051f1dc__wakuorg_nwaku:latest.log