265 lines
45 KiB
Plaintext

2026-03-05 04:34:31.418 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-05 04:34:31.420 DEBUG [tests.conftest] Running test: test_content_topic_also_in_docker_flags[/toychat/2/huilong/proto] with id: 2026-03-05_04-34-31__c1cd9ced-5502-4ee5-ab37-40c6087be240
2026-03-05 04:34:31.420 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-05 04:34:31.422 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-05 04:34:31.422 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-05 04:34:31.432 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:34:31.433 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-05_04-34-31__c1cd9ced-5502-4ee5-ab37-40c6087be240__wakuorg_nwaku:latest.log
2026-03-05 04:34:31.433 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:34:31.433 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:34:31.435 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:34:31.435 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.232.19
2026-03-05 04:34:31.435 DEBUG [src.node.docker_mananger] Generated ports ['29289', '29290', '29291', '29292', '29293']
2026-03-05 04:34:31.435 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-05 04:34:31.435 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-05 04:34:31.436 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:34:31.436 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:34:31.436 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:34:31.436 DEBUG [src.node.docker_mananger] docker run -i -t -p 29289:29289 -p 29290:29290 -p 29291:29291 -p 29292:29292 -p 29293:29293 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=29291 --rest-port=29289 --tcp-port=29290 --discv5-udp-port=29292 --rest-address=0.0.0.0 --nat=extip:172.18.232.19 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=8b5de3e93a2bc0deb7e2501988b6e9b0fbf81fbb5631acd4fb6ce9d6678f9d91 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=29293 --metrics-logging=true --relay=true --filter=true --content-topic=/toychat/2/huilong/proto
2026-03-05 04:34:31.623 ERROR [src.node.docker_mananger] Max retries reached for container 3d5ee4558476. Exiting log stream.
2026-03-05 04:34:31.636 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.232.19 waku 3bda26de4bd09dbc5f37cd1c5c896f2b4dc32999b82cf6aa0d7b48cd118fd311
2026-03-05 04:34:31.674 DEBUG [src.node.docker_mananger] Container started with ID 3bda26de4bd0. Setting up logs at ./log/docker/node1_2026-03-05_04-34-31__c1cd9ced-5502-4ee5-ab37-40c6087be240__wakuorg_nwaku:latest.log
2026-03-05 04:34:31.675 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 29289
2026-03-05 04:34:31.675 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:34:32.247 ERROR [src.node.docker_mananger] Max retries reached for container 441c758a8d40. Exiting log stream.
2026-03-05 04:34:32.675 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29289/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:32.678 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:32.778 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29289/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:32.781 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:32.881 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29289/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:32.884 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:32.984 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29289/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:32.987 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:33.087 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29289/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:33.091 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:33.191 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29289/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:33.242 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:33.343 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29289/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:33.347 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:34:33.347 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:34:33.348 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29289/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:33.351 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.232.19/tcp/29290/p2p/16Uiu2HAm17Tdr3wWYyAoCXEBUNLUm6F6LgLL176CnTLiVe1EtWrz","/ip4/172.18.232.19/tcp/29291/ws/p2p/16Uiu2HAm17Tdr3wWYyAoCXEBUNLUm6F6LgLL176CnTLiVe1EtWrz"],"enrUri":"enr:-L24QHRyLG1bzzpMrXT3TzOh47Kdvu95ZHyhgiI0y7WmczpfAvxPY-fY2miHb2K_YnX33d-CEjkkk7Dp3q3x0jE9Da4CgmlkgnY0gmlwhKwS6BOKbXVsdGlhZGRyc5YACASsEugTBnJqAAoErBLoEwZya90DgnJzhQACAQAAiXNlY3AyNTZrMaECVHu1rgtifhw_xCitgBMAsB8SmvKgh7Ig2GupLGuWLG-DdGNwgnJqg3VkcIJybIV3YWt1MgU"}'
2026-03-05 04:34:33.352 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:34:33.362 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:34:33.362 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-05_04-34-31__c1cd9ced-5502-4ee5-ab37-40c6087be240__wakuorg_nwaku:latest.log
2026-03-05 04:34:33.362 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:34:33.362 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:34:33.365 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:34:33.366 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.68.189
2026-03-05 04:34:33.366 DEBUG [src.node.docker_mananger] Generated ports ['2656', '2657', '2658', '2659', '2660']
2026-03-05 04:34:33.367 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-05 04:34:33.367 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-05 04:34:33.367 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:34:33.367 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:34:33.367 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:34:33.368 DEBUG [src.node.docker_mananger] docker run -i -t -p 2656:2656 -p 2657:2657 -p 2658:2658 -p 2659:2659 -p 2660:2660 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=2658 --rest-port=2656 --tcp-port=2657 --discv5-udp-port=2659 --rest-address=0.0.0.0 --nat=extip:172.18.68.189 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=f0ccbfacaf0ed9bcce065dacf77b78c8df6ec39b7e29aaab146dccd0b4e3a57e --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=2660 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QHRyLG1bzzpMrXT3TzOh47Kdvu95ZHyhgiI0y7WmczpfAvxPY-fY2miHb2K_YnX33d-CEjkkk7Dp3q3x0jE9Da4CgmlkgnY0gmlwhKwS6BOKbXVsdGlhZGRyc5YACASsEugTBnJqAAoErBLoEwZya90DgnJzhQACAQAAiXNlY3AyNTZrMaECVHu1rgtifhw_xCitgBMAsB8SmvKgh7Ig2GupLGuWLG-DdGNwgnJqg3VkcIJybIV3YWt1MgU --content-topic=/toychat/2/huilong/proto
2026-03-05 04:34:33.569 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.68.189 waku c0ce4825b301d00ebe25e7c9be0d49357cb1fd06e69a970dca0bfe3f5b66c90e
2026-03-05 04:34:33.605 DEBUG [src.node.docker_mananger] Container started with ID c0ce4825b301. Setting up logs at ./log/docker/node2_2026-03-05_04-34-31__c1cd9ced-5502-4ee5-ab37-40c6087be240__wakuorg_nwaku:latest.log
2026-03-05 04:34:33.606 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 2656
2026-03-05 04:34:33.606 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:34:34.606 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2656/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:34.609 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:34:34.610 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:34:34.610 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2656/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:34.612 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.68.189/tcp/2657/p2p/16Uiu2HAmNNVS8ooRzyEMnmKg3LSrwv72tmBjRKUqxJnF1YSHUNA9","/ip4/172.18.68.189/tcp/2658/ws/p2p/16Uiu2HAmNNVS8ooRzyEMnmKg3LSrwv72tmBjRKUqxJnF1YSHUNA9"],"enrUri":"enr:-L24QDmc212ut1pUL3JNOo_df3JlTSGOQvbwHZ7eRGHayPW6bvlCY28okQdjx_n7A3o8fMwd1QiocJHB3KJB2lOQECoCgmlkgnY0gmlwhKwSRL2KbXVsdGlhZGRyc5YACASsEkS9BgphAAoErBJEvQYKYt0DgnJzhQACAQAAiXNlY3AyNTZrMaEDkFrhYGS0E8vNV3AtVS_WRPqccLQKnuCwCQaOWvd6Q56DdGNwggphg3VkcIIKY4V3YWt1MgE"}'
2026-03-05 04:34:34.612 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:34:34.613 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:2656/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.232.19/tcp/29290/p2p/16Uiu2HAm17Tdr3wWYyAoCXEBUNLUm6F6LgLL176CnTLiVe1EtWrz"]'
2026-03-05 04:34:34.635 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:34:34.638 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29289/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-05 04:34:34.647 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:34:34.647 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:2656/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-05 04:34:34.656 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:34:34.658 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:29289/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 04:34:34.672 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:34:34.672 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-05 04:34:34.773 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-05 04:34:34.773 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:29289/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:34.776 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772685273771170899,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772685271337804012,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772685274658224869,"ephemeral":false,"proof":""}]'
2026-03-05 04:34:34.785 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-05 04:34:34.786 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-05 04:34:34.786 DEBUG [src.node.waku_node] Stopping container with id 3bda26de4bd0
2026-03-05 04:34:35.392 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:34:35.392 DEBUG [src.node.waku_node] Stopping container with id c0ce4825b301
2026-03-05 04:34:35.992 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:34:35.995 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-05 04:34:36.058 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:34:36.089 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:34:36.090 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-05 04:34:36.090 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-05 04:34:36.091 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-05_04-34-31__c1cd9ced-5502-4ee5-ab37-40c6087be240__wakuorg_nwaku:latest.log
2026-03-05 04:34:36.092 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-05_04-34-31__c1cd9ced-5502-4ee5-ab37-40c6087be240__wakuorg_nwaku:latest.log
2026-03-05 04:34:36.096 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-05 04:34:36.097 DEBUG [tests.conftest] Running test: test_content_topic_also_in_docker_flags[/toychat/2/huilong/proto] with id: 2026-03-05_04-34-36__5b55bd23-374c-4fc5-b303-c8193a123205
2026-03-05 04:34:36.097 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-05 04:34:36.097 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-05 04:34:36.098 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-05 04:34:36.106 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:34:36.106 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-05_04-34-36__5b55bd23-374c-4fc5-b303-c8193a123205__wakuorg_nwaku:latest.log
2026-03-05 04:34:36.106 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:34:36.106 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:34:36.109 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:34:36.109 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.177.63
2026-03-05 04:34:36.109 DEBUG [src.node.docker_mananger] Generated ports ['50490', '50491', '50492', '50493', '50494']
2026-03-05 04:34:36.109 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-05 04:34:36.109 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-05 04:34:36.109 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:34:36.109 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:34:36.110 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:34:36.110 DEBUG [src.node.docker_mananger] docker run -i -t -p 50490:50490 -p 50491:50491 -p 50492:50492 -p 50493:50493 -p 50494:50494 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=50492 --rest-port=50490 --tcp-port=50491 --discv5-udp-port=50493 --rest-address=0.0.0.0 --nat=extip:172.18.177.63 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=e0e4fdbbeed81abe8bf2fbafe9be4734bc0ab4f462ca45dee2e256e4008e9ac0 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=50494 --metrics-logging=true --relay=true --filter=true --content-topic=/toychat/2/huilong/proto
2026-03-05 04:34:36.305 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.177.63 waku 81f3c7fb032d2894c530685b71a2595adce100253417f88df90e7ce7214ce636
2026-03-05 04:34:36.340 DEBUG [src.node.docker_mananger] Container started with ID 81f3c7fb032d. Setting up logs at ./log/docker/node1_2026-03-05_04-34-36__5b55bd23-374c-4fc5-b303-c8193a123205__wakuorg_nwaku:latest.log
2026-03-05 04:34:36.340 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 50490
2026-03-05 04:34:36.341 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:34:36.346 ERROR [src.node.docker_mananger] Max retries reached for container 3bda26de4bd0. Exiting log stream.
2026-03-05 04:34:36.809 ERROR [src.node.docker_mananger] Max retries reached for container c0ce4825b301. Exiting log stream.
2026-03-05 04:34:37.341 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50490/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:37.344 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:37.445 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50490/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:37.487 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:37.588 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50490/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:37.591 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:34:37.591 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:34:37.591 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50490/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:37.596 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.177.63/tcp/50491/p2p/16Uiu2HAm37EU5uv7um8e4fz5x1MPhnkHv4CvJdkUtLkqU71Jrbj3","/ip4/172.18.177.63/tcp/50492/ws/p2p/16Uiu2HAm37EU5uv7um8e4fz5x1MPhnkHv4CvJdkUtLkqU71Jrbj3"],"enrUri":"enr:-L24QCP6xcLndI33nz67EVnvBH09iM3vmrTX4yhQeCzMBDGSc7oEEIBJaGvlIXJtqRnpCHCNoLts-1bfS-gGm7yWJ5gCgmlkgnY0gmlwhKwSsT-KbXVsdGlhZGRyc5YACASsErE_BsU7AAoErBKxPwbFPN0DgnJzhQACAQAAiXNlY3AyNTZrMaECciQxsL0SxFUPyl5_CzfDdp3o5zlw2ebvh5Cp1OwV6waDdGNwgsU7g3VkcILFPYV3YWt1MgU"}'
2026-03-05 04:34:37.597 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:34:37.606 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:34:37.606 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-05_04-34-36__5b55bd23-374c-4fc5-b303-c8193a123205__wakuorg_nwaku:latest.log
2026-03-05 04:34:37.606 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:34:37.607 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:34:37.608 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:34:37.608 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.251.72
2026-03-05 04:34:37.608 DEBUG [src.node.docker_mananger] Generated ports ['48337', '48338', '48339', '48340', '48341']
2026-03-05 04:34:37.609 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-05 04:34:37.609 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-05 04:34:37.609 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:34:37.609 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:34:37.609 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:34:37.609 DEBUG [src.node.docker_mananger] docker run -i -t -p 48337:48337 -p 48338:48338 -p 48339:48339 -p 48340:48340 -p 48341:48341 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=48339 --rest-port=48337 --tcp-port=48338 --discv5-udp-port=48340 --rest-address=0.0.0.0 --nat=extip:172.18.251.72 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=d3e1c1b5afe614da5eafe51c1c53e3c21e71adc410db3ad7db228d96ffc45b5b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=48341 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QCP6xcLndI33nz67EVnvBH09iM3vmrTX4yhQeCzMBDGSc7oEEIBJaGvlIXJtqRnpCHCNoLts-1bfS-gGm7yWJ5gCgmlkgnY0gmlwhKwSsT-KbXVsdGlhZGRyc5YACASsErE_BsU7AAoErBKxPwbFPN0DgnJzhQACAQAAiXNlY3AyNTZrMaECciQxsL0SxFUPyl5_CzfDdp3o5zlw2ebvh5Cp1OwV6waDdGNwgsU7g3VkcILFPYV3YWt1MgU --content-topic=/toychat/2/huilong/proto
2026-03-05 04:34:37.806 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.251.72 waku 254d7d9cb01cd32b70eecd7e1823ea925db0b5b8c1e3a3e48bfec3fc7776551c
2026-03-05 04:34:37.844 DEBUG [src.node.docker_mananger] Container started with ID 254d7d9cb01c. Setting up logs at ./log/docker/node2_2026-03-05_04-34-36__5b55bd23-374c-4fc5-b303-c8193a123205__wakuorg_nwaku:latest.log
2026-03-05 04:34:37.844 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 48337
2026-03-05 04:34:37.844 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:34:38.845 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:48337/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:38.848 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:34:38.849 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:34:38.850 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:48337/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:38.853 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.251.72/tcp/48338/p2p/16Uiu2HAmHd2K34ESzBbj2w9iiHyw2egxiGX2AEfPB3JyQmXmTyxQ","/ip4/172.18.251.72/tcp/48339/ws/p2p/16Uiu2HAmHd2K34ESzBbj2w9iiHyw2egxiGX2AEfPB3JyQmXmTyxQ"],"enrUri":"enr:-L24QAeGYeEQfbf-1o9vHhirIA4B2ZzP6_vzkhxEr08M0464P6k8dCSlacJduayUJxIhTsN8fBpDTGHr1q5TrYrjQ54CgmlkgnY0gmlwhKwS-0iKbXVsdGlhZGRyc5YACASsEvtIBrzSAAoErBL7SAa8090DgnJzhQACAQAAiXNlY3AyNTZrMaEDScl9kIteiRaatOq68VSF62gm91F1N-KZPOI2O0QMSn2DdGNwgrzSg3VkcIK81IV3YWt1MgE"}'
2026-03-05 04:34:38.854 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:34:38.855 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48337/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.177.63/tcp/50491/p2p/16Uiu2HAm37EU5uv7um8e4fz5x1MPhnkHv4CvJdkUtLkqU71Jrbj3"]'
2026-03-05 04:34:38.892 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:34:38.896 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:50490/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-05 04:34:38.903 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:34:38.905 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:48337/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-05 04:34:38.909 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:34:38.910 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:50490/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 04:34:38.930 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:34:38.930 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-05 04:34:39.030 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-05 04:34:39.031 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50490/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:39.034 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772685275240035270,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772685278853493844,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772685278910100495,"ephemeral":false,"proof":""}]'
2026-03-05 04:34:39.042 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-05 04:34:39.043 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-05 04:34:39.043 DEBUG [src.node.waku_node] Stopping container with id 81f3c7fb032d
2026-03-05 04:34:39.633 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:34:39.633 DEBUG [src.node.waku_node] Stopping container with id 254d7d9cb01c
2026-03-05 04:34:40.229 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:34:40.230 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-05 04:34:40.275 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:34:40.311 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:34:40.312 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-05 04:34:40.313 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-05 04:34:40.314 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-05_04-34-36__5b55bd23-374c-4fc5-b303-c8193a123205__wakuorg_nwaku:latest.log
2026-03-05 04:34:40.315 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-05_04-34-36__5b55bd23-374c-4fc5-b303-c8193a123205__wakuorg_nwaku:latest.log
2026-03-05 04:34:40.319 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-05 04:34:40.319 DEBUG [tests.conftest] Running test: test_content_topic_also_in_docker_flags[/toychat/2/huilong/proto] with id: 2026-03-05_04-34-40__5e34d9fb-a0a6-4768-b038-cd5fe6d8474c
2026-03-05 04:34:40.319 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-05 04:34:40.320 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-05 04:34:40.320 DEBUG [src.steps.sharding] Running fixture setup: sharding_setup
2026-03-05 04:34:40.327 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:34:40.328 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-05_04-34-40__5e34d9fb-a0a6-4768-b038-cd5fe6d8474c__wakuorg_nwaku:latest.log
2026-03-05 04:34:40.328 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:34:40.328 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:34:40.329 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:34:40.330 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.11.127
2026-03-05 04:34:40.330 DEBUG [src.node.docker_mananger] Generated ports ['34813', '34814', '34815', '34816', '34817']
2026-03-05 04:34:40.330 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-05 04:34:40.330 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-05 04:34:40.330 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:34:40.330 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:34:40.330 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:34:40.331 DEBUG [src.node.docker_mananger] docker run -i -t -p 34813:34813 -p 34814:34814 -p 34815:34815 -p 34816:34816 -p 34817:34817 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=34815 --rest-port=34813 --tcp-port=34814 --discv5-udp-port=34816 --rest-address=0.0.0.0 --nat=extip:172.18.11.127 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=5c8f53f8bfc49adec2ffba88cdab875c3ce59e9e64ef0fa01b827fffd4744afe --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=34817 --metrics-logging=true --relay=true --filter=true --content-topic=/toychat/2/huilong/proto
2026-03-05 04:34:40.516 ERROR [src.node.docker_mananger] Max retries reached for container 81f3c7fb032d. Exiting log stream.
2026-03-05 04:34:40.527 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.11.127 waku 2980925be47f0ed470c641e355a1b4c2e6551c54b5c06259b429e78cca2ccac0
2026-03-05 04:34:40.564 DEBUG [src.node.docker_mananger] Container started with ID 2980925be47f. Setting up logs at ./log/docker/node1_2026-03-05_04-34-40__5e34d9fb-a0a6-4768-b038-cd5fe6d8474c__wakuorg_nwaku:latest.log
2026-03-05 04:34:40.564 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 34813
2026-03-05 04:34:40.564 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:34:41.086 ERROR [src.node.docker_mananger] Max retries reached for container 254d7d9cb01c. Exiting log stream.
2026-03-05 04:34:41.566 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:34813/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:41.569 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:34:41.570 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:34:41.570 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:34813/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:41.572 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.11.127/tcp/34814/p2p/16Uiu2HAmE6tVQa3eLjptzqgLssyQURUz5qRLURtadV69aVTXpcdU","/ip4/172.18.11.127/tcp/34815/ws/p2p/16Uiu2HAmE6tVQa3eLjptzqgLssyQURUz5qRLURtadV69aVTXpcdU"],"enrUri":"enr:-L24QPFRaxaoAeIinSTMDMoUf2_I9S86oQ13fom8zgbhBtEaPQQYH_wWaC3ntjm3WGw_3YNZmBCC-zJqL1ojMAtMICUCgmlkgnY0gmlwhKwSC3-KbXVsdGlhZGRyc5YACASsEgt_Bof-AAoErBILfwaH_90DgnJzhQACAQAAiXNlY3AyNTZrMaEDFX4pd_hCihcp8dr5mYkDmiYE7KbzHPQfk5Su7fzrDyeDdGNwgof-g3VkcIKIAIV3YWt1MgU"}'
2026-03-05 04:34:41.572 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:34:41.580 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:34:41.580 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-05_04-34-40__5e34d9fb-a0a6-4768-b038-cd5fe6d8474c__wakuorg_nwaku:latest.log
2026-03-05 04:34:41.580 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:34:41.580 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:34:41.582 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:34:41.582 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.30.159
2026-03-05 04:34:41.582 DEBUG [src.node.docker_mananger] Generated ports ['36412', '36413', '36414', '36415', '36416']
2026-03-05 04:34:41.582 DEBUG [src.node.waku_node] Cluster id was resolved to: 2
2026-03-05 04:34:41.582 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-03-05 04:34:41.582 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:34:41.583 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:34:41.583 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:34:41.583 DEBUG [src.node.docker_mananger] docker run -i -t -p 36412:36412 -p 36413:36413 -p 36414:36414 -p 36415:36415 -p 36416:36416 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=36414 --rest-port=36412 --tcp-port=36413 --discv5-udp-port=36415 --rest-address=0.0.0.0 --nat=extip:172.18.30.159 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=25c7ccbf63ddcc68e5ae4de2a55cc2a7d589aced13fb57a143dff3dedffc428b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=36416 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPFRaxaoAeIinSTMDMoUf2_I9S86oQ13fom8zgbhBtEaPQQYH_wWaC3ntjm3WGw_3YNZmBCC-zJqL1ojMAtMICUCgmlkgnY0gmlwhKwSC3-KbXVsdGlhZGRyc5YACASsEgt_Bof-AAoErBILfwaH_90DgnJzhQACAQAAiXNlY3AyNTZrMaEDFX4pd_hCihcp8dr5mYkDmiYE7KbzHPQfk5Su7fzrDyeDdGNwgof-g3VkcIKIAIV3YWt1MgU --content-topic=/toychat/2/huilong/proto
2026-03-05 04:34:41.783 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.30.159 waku 3abafbcd9c5e1dfa48de0476e9c742e999d75b29f1e4cfb5e4879d5138620823
2026-03-05 04:34:41.818 DEBUG [src.node.docker_mananger] Container started with ID 3abafbcd9c5e. Setting up logs at ./log/docker/node2_2026-03-05_04-34-40__5e34d9fb-a0a6-4768-b038-cd5fe6d8474c__wakuorg_nwaku:latest.log
2026-03-05 04:34:41.819 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 36412
2026-03-05 04:34:41.819 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:34:42.819 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36412/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:42.822 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:42.922 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36412/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:42.925 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:43.025 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36412/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:43.028 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:43.128 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36412/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:43.131 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:43.231 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36412/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:43.234 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:43.335 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36412/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:43.337 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:43.438 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36412/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:43.462 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
2026-03-05 04:34:43.562 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36412/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:43.566 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Connected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:34:43.566 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:34:43.567 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36412/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:43.570 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.30.159/tcp/36413/p2p/16Uiu2HAmKUHk7uEFAqLnUDoo97uu9FTNiJ4za9Tx3onfwhgqJ7zV","/ip4/172.18.30.159/tcp/36414/ws/p2p/16Uiu2HAmKUHk7uEFAqLnUDoo97uu9FTNiJ4za9Tx3onfwhgqJ7zV"],"enrUri":"enr:-L24QBaR0XdQP877gI_xFzOmpZthjK4NEjArVTqO1JZTbNQ6VrVAyoneeFKIsfRHsa2hlN0Xwk0DrLL2L2k3q-Etra8CgmlkgnY0gmlwhKwSHp-KbXVsdGlhZGRyc5YACASsEh6fBo49AAoErBIenwaOPt0DgnJzhQACAQAAiXNlY3AyNTZrMaEDZUQWLD4HBkiLVQntiI1opIvW8PN6LOXNe7nKpLRz1-aDdGNwgo49g3VkcIKOP4V3YWt1MgE"}'
2026-03-05 04:34:43.570 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:34:43.570 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:36412/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.11.127/tcp/34814/p2p/16Uiu2HAmE6tVQa3eLjptzqgLssyQURUz5qRLURtadV69aVTXpcdU"]'
2026-03-05 04:34:43.592 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:34:43.594 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:34813/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-05 04:34:43.605 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:34:43.607 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:36412/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/2/0"]'
2026-03-05 04:34:43.614 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:34:43.614 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:34813/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 04:34:43.626 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:34:43.627 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-05 04:34:43.727 DEBUG [src.steps.sharding] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-05 04:34:43.727 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:34813/relay/v1/messages/%2Fwaku%2F2%2Frs%2F2%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:34:43.730 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772685279368375619,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772685278910100495,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772685278853493844,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772685283614688852,"ephemeral":false,"proof":""},{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772685283593138823,"ephemeral":false,"proof":""}]'
2026-03-05 04:34:43.738 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-05 04:34:43.740 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-05 04:34:43.740 DEBUG [src.node.waku_node] Stopping container with id 2980925be47f
2026-03-05 04:34:44.353 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:34:44.353 DEBUG [src.node.waku_node] Stopping container with id 3abafbcd9c5e
2026-03-05 04:34:44.920 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:34:44.922 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-05 04:34:44.959 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:34:44.992 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:34:44.994 DEBUG [tests.conftest] Running fixture teardown: attach_logs_on_fail
2026-03-05 04:34:44.995 DEBUG [tests.conftest] Test failed, attempting to attach logs to the allure reports
2026-03-05 04:34:44.997 DEBUG [src.libs.common] Attaching file ./log/docker/node2_2026-03-05_04-34-40__5e34d9fb-a0a6-4768-b038-cd5fe6d8474c__wakuorg_nwaku:latest.log
2026-03-05 04:34:45.001 DEBUG [src.libs.common] Attaching file ./log/docker/node1_2026-03-05_04-34-40__5e34d9fb-a0a6-4768-b038-cd5fe6d8474c__wakuorg_nwaku:latest.log