192 lines
35 KiB
Plaintext

2026-03-02 04:35:30.418 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-02 04:35:30.420 DEBUG [tests.conftest] Running test: test_last_node_to_start_publishes with id: 2026-03-02_04-35-30__7b44371c-a980-433f-826e-bc445382f094
2026-03-02 04:35:30.420 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-02 04:35:30.421 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-02 04:35:30.421 DEBUG [src.steps.relay] Running fixture setup: setup_main_relay_nodes
2026-03-02 04:35:30.430 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-02 04:35:30.430 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-02_04-35-30__7b44371c-a980-433f-826e-bc445382f094__wakuorg_nwaku:latest.log
2026-03-02 04:35:30.431 DEBUG [src.node.waku_node] Starting Node...
2026-03-02 04:35:30.431 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-02 04:35:30.434 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-02 04:35:30.435 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.88.16
2026-03-02 04:35:30.435 DEBUG [src.node.docker_mananger] Generated ports ['42004', '42005', '42006', '42007', '42008']
2026-03-02 04:35:30.435 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-02 04:35:30.435 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-02 04:35:30.436 DEBUG [src.node.waku_node] Using volumes []
2026-03-02 04:35:30.436 DEBUG [src.node.docker_mananger] docker run -i -t -p 42004:42004 -p 42005:42005 -p 42006:42006 -p 42007:42007 -p 42008:42008 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=42006 --rest-port=42004 --tcp-port=42005 --discv5-udp-port=42007 --rest-address=0.0.0.0 --nat=extip:172.18.88.16 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=b94e65bfd3ef2acaed86d277cce80f2eac3ddfc06810e22b8adcd12f9ad9e04e --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=42008 --metrics-logging=true --relay=true
2026-03-02 04:35:30.612 ERROR [src.node.docker_mananger] Max retries reached for container 389728232a6d. Exiting log stream.
2026-03-02 04:35:30.639 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.88.16 waku 7a3a4772e97b8f25eff01b15ce86762db126c9b467611304afa89e88afccfd2a
2026-03-02 04:35:30.674 DEBUG [src.node.docker_mananger] Container started with ID 7a3a4772e97b. Setting up logs at ./log/docker/node1_2026-03-02_04-35-30__7b44371c-a980-433f-826e-bc445382f094__wakuorg_nwaku:latest.log
2026-03-02 04:35:30.674 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 42004
2026-03-02 04:35:30.675 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-02 04:35:31.247 ERROR [src.node.docker_mananger] Max retries reached for container 872d77efc875. Exiting log stream.
2026-03-02 04:35:31.675 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42004/health" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:31.678 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-02 04:35:31.679 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-02 04:35:31.679 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42004/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:31.681 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.88.16/tcp/42005/p2p/16Uiu2HAmNSWn44DX9GbXuUaypBXF64QGjARTVaWCQyQZP6gg1cNy","/ip4/172.18.88.16/tcp/42006/ws/p2p/16Uiu2HAmNSWn44DX9GbXuUaypBXF64QGjARTVaWCQyQZP6gg1cNy"],"enrUri":"enr:-L24QL5t9Un6ClzncccpBQTaahU4cMx4nf7a6Ikc6WwkIrLZT2Jv238Qw1V0AF7OOp-LkxssaockAIjzlMdk3E0jrlICgmlkgnY0gmlwhKwSWBCKbXVsdGlhZGRyc5YACASsElgQBqQVAAoErBJYEAakFt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDkWK4_XnsU44jYI_UEzHgtyH1ShFaytHP7xwJWYy1XfaDdGNwgqQVg3VkcIKkF4V3YWt1MgE"}'
2026-03-02 04:35:31.682 INFO [src.node.waku_node] REST service is ready !!
2026-03-02 04:35:31.688 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-02 04:35:31.688 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-02_04-35-30__7b44371c-a980-433f-826e-bc445382f094__wakuorg_nwaku:latest.log
2026-03-02 04:35:31.689 DEBUG [src.node.waku_node] Starting Node...
2026-03-02 04:35:31.689 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-02 04:35:31.690 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-02 04:35:31.690 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.95.36
2026-03-02 04:35:31.690 DEBUG [src.node.docker_mananger] Generated ports ['16243', '16244', '16245', '16246', '16247']
2026-03-02 04:35:31.691 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-02 04:35:31.691 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-02 04:35:31.691 DEBUG [src.node.waku_node] Using volumes []
2026-03-02 04:35:31.691 DEBUG [src.node.docker_mananger] docker run -i -t -p 16243:16243 -p 16244:16244 -p 16245:16245 -p 16246:16246 -p 16247:16247 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=16245 --rest-port=16243 --tcp-port=16244 --discv5-udp-port=16246 --rest-address=0.0.0.0 --nat=extip:172.18.95.36 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=d5c98f409d44cfc162e4ddfac42f057eacf39fdbfede6f40f20b844adb32c1e0 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=16247 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QL5t9Un6ClzncccpBQTaahU4cMx4nf7a6Ikc6WwkIrLZT2Jv238Qw1V0AF7OOp-LkxssaockAIjzlMdk3E0jrlICgmlkgnY0gmlwhKwSWBCKbXVsdGlhZGRyc5YACASsElgQBqQVAAoErBJYEAakFt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDkWK4_XnsU44jYI_UEzHgtyH1ShFaytHP7xwJWYy1XfaDdGNwgqQVg3VkcIKkF4V3YWt1MgE
2026-03-02 04:35:31.889 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.95.36 waku 82c44f037a621a414617930a78a050bed951820cf1c329b10d3bcf0bfd208718
2026-03-02 04:35:31.927 DEBUG [src.node.docker_mananger] Container started with ID 82c44f037a62. Setting up logs at ./log/docker/node2_2026-03-02_04-35-30__7b44371c-a980-433f-826e-bc445382f094__wakuorg_nwaku:latest.log
2026-03-02 04:35:31.927 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 16243
2026-03-02 04:35:31.927 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-02 04:35:32.928 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16243/health" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:32.940 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-02 04:35:32.945 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-02 04:35:32.945 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16243/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:32.950 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.95.36/tcp/16244/p2p/16Uiu2HAm9imBcdEXhBA5aK3Dy5PWPa8ruvGW2nsLWwfMaKQ24BQe","/ip4/172.18.95.36/tcp/16245/ws/p2p/16Uiu2HAm9imBcdEXhBA5aK3Dy5PWPa8ruvGW2nsLWwfMaKQ24BQe"],"enrUri":"enr:-L24QGxX5vbZgtdc3IQj80dqKkMUknAMgTnTDXKV6YNfQLoJBcITc-lQsDIkEwGFTOvIRe13DnVt3GzPe3lPAPY6AmECgmlkgnY0gmlwhKwSXySKbXVsdGlhZGRyc5YACASsEl8kBj90AAoErBJfJAY_dd0DgnJzhQADAQAAiXNlY3AyNTZrMaEC1GRf7F4ZowZnEhr6GPmdAv4rnI8ZkPB_eGpMzsgdoKuDdGNwgj90g3VkcII_doV3YWt1MgE"}'
2026-03-02 04:35:32.951 INFO [src.node.waku_node] REST service is ready !!
2026-03-02 04:35:32.952 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16243/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.88.16/tcp/42005/p2p/16Uiu2HAmNSWn44DX9GbXuUaypBXF64QGjARTVaWCQyQZP6gg1cNy"]'
2026-03-02 04:35:32.955 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:35:32.955 DEBUG [src.steps.relay] Running fixture setup: setup_optional_relay_nodes
2026-03-02 04:35:32.962 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-02 04:35:32.962 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-02_04-35-30__7b44371c-a980-433f-826e-bc445382f094__wakuorg_nwaku:latest.log
2026-03-02 04:35:32.963 DEBUG [src.node.waku_node] Starting Node...
2026-03-02 04:35:32.963 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-02 04:35:32.964 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-02 04:35:32.964 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.10.159
2026-03-02 04:35:32.965 DEBUG [src.node.docker_mananger] Generated ports ['1174', '1175', '1176', '1177', '1178']
2026-03-02 04:35:32.965 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-02 04:35:32.965 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-02 04:35:32.965 DEBUG [src.node.waku_node] Using volumes []
2026-03-02 04:35:32.965 DEBUG [src.node.docker_mananger] docker run -i -t -p 1174:1174 -p 1175:1175 -p 1176:1176 -p 1177:1177 -p 1178:1178 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=1176 --rest-port=1174 --tcp-port=1175 --discv5-udp-port=1177 --rest-address=0.0.0.0 --nat=extip:172.18.10.159 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=efebe3e68fbbbeeedd4d8deb5cbf0a54b29e03acbd505d5cbbc0deeec79f0ded --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=1178 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QL5t9Un6ClzncccpBQTaahU4cMx4nf7a6Ikc6WwkIrLZT2Jv238Qw1V0AF7OOp-LkxssaockAIjzlMdk3E0jrlICgmlkgnY0gmlwhKwSWBCKbXVsdGlhZGRyc5YACASsElgQBqQVAAoErBJYEAakFt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDkWK4_XnsU44jYI_UEzHgtyH1ShFaytHP7xwJWYy1XfaDdGNwgqQVg3VkcIKkF4V3YWt1MgE
2026-03-02 04:35:33.162 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.10.159 waku 8237743a5f134e4b96a7406513de1e6dd4413d81636817bc1409db14cd6353d6
2026-03-02 04:35:33.200 DEBUG [src.node.docker_mananger] Container started with ID 8237743a5f13. Setting up logs at ./log/docker/node3_2026-03-02_04-35-30__7b44371c-a980-433f-826e-bc445382f094__wakuorg_nwaku:latest.log
2026-03-02 04:35:33.200 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 1174
2026-03-02 04:35:33.201 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-02 04:35:34.201 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1174/health" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:34.213 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-02 04:35:34.214 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-02 04:35:34.214 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1174/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:34.222 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.10.159/tcp/1175/p2p/16Uiu2HAm9YSKZaNpBhK5McujmMAuqFJECxawoL99e4xrvVtjkvSk","/ip4/172.18.10.159/tcp/1176/ws/p2p/16Uiu2HAm9YSKZaNpBhK5McujmMAuqFJECxawoL99e4xrvVtjkvSk"],"enrUri":"enr:-L24QCb8h0pam05KoNlwxxXiwHsMlwxJXPYqA2miIgSR-mG9TSMxizTtVHCsZVf9WgpdGJnFbaCFgsQpZEmVb0Ux5MACgmlkgnY0gmlwhKwSCp-KbXVsdGlhZGRyc5YACASsEgqfBgSXAAoErBIKnwYEmN0DgnJzhQADAQAAiXNlY3AyNTZrMaEC0b88-RAR9PNQ1N8ABYd6pxvd5j26khLzEgsQ5mE0TQGDdGNwggSXg3VkcIIEmYV3YWt1MgE"}'
2026-03-02 04:35:34.222 INFO [src.node.waku_node] REST service is ready !!
2026-03-02 04:35:34.223 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:1174/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.88.16/tcp/42005/p2p/16Uiu2HAmNSWn44DX9GbXuUaypBXF64QGjARTVaWCQyQZP6gg1cNy"]'
2026-03-02 04:35:34.226 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:35:34.233 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-02 04:35:34.233 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-02_04-35-30__7b44371c-a980-433f-826e-bc445382f094__wakuorg_nwaku:latest.log
2026-03-02 04:35:34.234 DEBUG [src.node.waku_node] Starting Node...
2026-03-02 04:35:34.234 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-02 04:35:34.235 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-02 04:35:34.235 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.248.12
2026-03-02 04:35:34.235 DEBUG [src.node.docker_mananger] Generated ports ['6629', '6630', '6631', '6632', '6633']
2026-03-02 04:35:34.236 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-02 04:35:34.236 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-02 04:35:34.236 DEBUG [src.node.waku_node] Using volumes []
2026-03-02 04:35:34.236 DEBUG [src.node.docker_mananger] docker run -i -t -p 6629:6629 -p 6630:6630 -p 6631:6631 -p 6632:6632 -p 6633:6633 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=6631 --rest-port=6629 --tcp-port=6630 --discv5-udp-port=6632 --rest-address=0.0.0.0 --nat=extip:172.18.248.12 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=f62d61a79ea8c26e97b4d5f11e00b6bce5875a35309597a8bac9f8f1efadaa2f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=6633 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QL5t9Un6ClzncccpBQTaahU4cMx4nf7a6Ikc6WwkIrLZT2Jv238Qw1V0AF7OOp-LkxssaockAIjzlMdk3E0jrlICgmlkgnY0gmlwhKwSWBCKbXVsdGlhZGRyc5YACASsElgQBqQVAAoErBJYEAakFt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDkWK4_XnsU44jYI_UEzHgtyH1ShFaytHP7xwJWYy1XfaDdGNwgqQVg3VkcIKkF4V3YWt1MgE
2026-03-02 04:35:34.453 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.248.12 waku 4dcc908fcff0b6ae13a6c929ca7762a0aa2f67d59810e7353f9aaa0aa3caa6e5
2026-03-02 04:35:34.488 DEBUG [src.node.docker_mananger] Container started with ID 4dcc908fcff0. Setting up logs at ./log/docker/node4_2026-03-02_04-35-30__7b44371c-a980-433f-826e-bc445382f094__wakuorg_nwaku:latest.log
2026-03-02 04:35:34.489 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 6629
2026-03-02 04:35:34.490 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-02 04:35:35.491 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6629/health" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:35.512 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-02 04:35:35.512 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-02 04:35:35.512 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6629/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:35.518 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.248.12/tcp/6630/p2p/16Uiu2HAmPhKj1gWs6ZHTfWyGeLe2G8nWFmTFKMtfyiWjE6E7vXth","/ip4/172.18.248.12/tcp/6631/ws/p2p/16Uiu2HAmPhKj1gWs6ZHTfWyGeLe2G8nWFmTFKMtfyiWjE6E7vXth"],"enrUri":"enr:-L24QHggmitwUc2wJboUGtwojc67D5vlbEnhUHR7O1i252rGPgN-kK7BJiBQDpqJYu1dyk1kYY2D5oqvu6kcEbx18P4CgmlkgnY0gmlwhKwS-AyKbXVsdGlhZGRyc5YACASsEvgMBhnmAAoErBL4DAYZ590DgnJzhQADAQAAiXNlY3AyNTZrMaEDpAmf6ZSBcShSGTg2BgxIqDyIGyfHgTycO_APac12D3aDdGNwghnmg3VkcIIZ6IV3YWt1MgE"}'
2026-03-02 04:35:35.519 INFO [src.node.waku_node] REST service is ready !!
2026-03-02 04:35:35.520 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6629/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.88.16/tcp/42005/p2p/16Uiu2HAmNSWn44DX9GbXuUaypBXF64QGjARTVaWCQyQZP6gg1cNy"]'
2026-03-02 04:35:35.525 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:35:35.532 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-02 04:35:35.532 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-03-02_04-35-30__7b44371c-a980-433f-826e-bc445382f094__wakuorg_nwaku:latest.log
2026-03-02 04:35:35.532 DEBUG [src.node.waku_node] Starting Node...
2026-03-02 04:35:35.532 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-02 04:35:35.534 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-02 04:35:35.534 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.178.69
2026-03-02 04:35:35.534 DEBUG [src.node.docker_mananger] Generated ports ['32815', '32816', '32817', '32818', '32819']
2026-03-02 04:35:35.534 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-02 04:35:35.534 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-02 04:35:35.534 DEBUG [src.node.waku_node] Using volumes []
2026-03-02 04:35:35.534 DEBUG [src.node.docker_mananger] docker run -i -t -p 32815:32815 -p 32816:32816 -p 32817:32817 -p 32818:32818 -p 32819:32819 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=32817 --rest-port=32815 --tcp-port=32816 --discv5-udp-port=32818 --rest-address=0.0.0.0 --nat=extip:172.18.178.69 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=cdfd6db76c636edd249172c65c6901fe1e4f7fd6c1f444e9ec6f9c1ba38ccfa0 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=32819 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QL5t9Un6ClzncccpBQTaahU4cMx4nf7a6Ikc6WwkIrLZT2Jv238Qw1V0AF7OOp-LkxssaockAIjzlMdk3E0jrlICgmlkgnY0gmlwhKwSWBCKbXVsdGlhZGRyc5YACASsElgQBqQVAAoErBJYEAakFt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDkWK4_XnsU44jYI_UEzHgtyH1ShFaytHP7xwJWYy1XfaDdGNwgqQVg3VkcIKkF4V3YWt1MgE
2026-03-02 04:35:35.744 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.178.69 waku d5c511e4fceeb5475809680acf657bb8f8e91487928fb379a210aa57572bc447
2026-03-02 04:35:35.779 DEBUG [src.node.docker_mananger] Container started with ID d5c511e4fcee. Setting up logs at ./log/docker/node5_2026-03-02_04-35-30__7b44371c-a980-433f-826e-bc445382f094__wakuorg_nwaku:latest.log
2026-03-02 04:35:35.780 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 32815
2026-03-02 04:35:35.780 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-02 04:35:36.782 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32815/health" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:36.798 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-02 04:35:36.801 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-02 04:35:36.802 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32815/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:36.810 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.178.69/tcp/32816/p2p/16Uiu2HAkzJo3YYpUyUqmjtwQHj2JLsYc1JeRnyy7SiACBVqekgsf","/ip4/172.18.178.69/tcp/32817/ws/p2p/16Uiu2HAkzJo3YYpUyUqmjtwQHj2JLsYc1JeRnyy7SiACBVqekgsf"],"enrUri":"enr:-L24QJX0FBa-kC03o2MfSTjyDPi-kKh_y2yupXMEOALT_CpBZpEjrRaAnzjQjIX6vvGZLXvxE1KvK6bOSZWMbmrOtmUCgmlkgnY0gmlwhKwSskWKbXVsdGlhZGRyc5YACASsErJFBoAwAAoErBKyRQaAMd0DgnJzhQADAQAAiXNlY3AyNTZrMaECSIdbZeRFz8gIcukMeCLNLaWt1B9RZgPagAUfFAznOv6DdGNwgoAwg3VkcIKAMoV3YWt1MgE"}'
2026-03-02 04:35:36.811 INFO [src.node.waku_node] REST service is ready !!
2026-03-02 04:35:36.811 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32815/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.88.16/tcp/42005/p2p/16Uiu2HAmNSWn44DX9GbXuUaypBXF64QGjARTVaWCQyQZP6gg1cNy"]'
2026-03-02 04:35:36.814 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:35:36.815 DEBUG [src.steps.relay] Running fixture setup: subscribe_main_relay_nodes
2026-03-02 04:35:36.815 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:42004/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-02 04:35:36.826 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:35:36.827 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:16243/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-02 04:35:36.832 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:35:36.834 DEBUG [src.steps.relay] Running fixture setup: subscribe_optional_relay_nodes
2026-03-02 04:35:36.837 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:1174/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-02 04:35:36.845 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:35:36.845 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6629/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-02 04:35:36.852 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:35:36.853 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32815/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-02 04:35:36.861 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:35:36.861 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:42004/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-02 04:35:36.875 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:35:36.876 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-02 04:35:36.976 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-02 04:35:36.976 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42004/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:36.980 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426136861822464,"ephemeral":false,"proof":""}]'
2026-03-02 04:35:36.981 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-02 04:35:36.981 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16243/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:36.984 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426136861822464,"ephemeral":false,"proof":""}]'
2026-03-02 04:35:36.985 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-03-02 04:35:36.985 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1174/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:36.988 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426136861822464,"ephemeral":false,"proof":""}]'
2026-03-02 04:35:36.990 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-03-02 04:35:36.991 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6629/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:36.993 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426136861822464,"ephemeral":false,"proof":""}]'
2026-03-02 04:35:36.994 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-03-02 04:35:36.994 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32815/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:36.997 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426136861822464,"ephemeral":false,"proof":""}]'
2026-03-02 04:35:36.998 INFO [src.steps.relay] WARM UP successful!!
2026-03-02 04:35:36.999 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32815/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-02 04:35:37.004 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-02 04:35:37.004 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-02 04:35:37.105 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-02 04:35:37.105 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42004/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:37.108 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426136999521051,"ephemeral":false,"proof":""}]'
2026-03-02 04:35:37.110 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-02 04:35:37.110 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:16243/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:37.113 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426136999521051,"ephemeral":false,"proof":""}]'
2026-03-02 04:35:37.114 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-03-02 04:35:37.114 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:1174/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:37.117 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426136999521051,"ephemeral":false,"proof":""}]'
2026-03-02 04:35:37.118 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-03-02 04:35:37.118 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6629/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:37.121 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426136999521051,"ephemeral":false,"proof":""}]'
2026-03-02 04:35:37.122 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-03-02 04:35:37.122 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32815/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-02 04:35:37.125 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772426136999521051,"ephemeral":false,"proof":""}]'
2026-03-02 04:35:37.129 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-02 04:35:37.130 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-02 04:35:37.130 DEBUG [src.node.waku_node] Stopping container with id 7a3a4772e97b
2026-03-02 04:35:37.716 DEBUG [src.node.waku_node] Container stopped.
2026-03-02 04:35:37.716 DEBUG [src.node.waku_node] Stopping container with id 82c44f037a62
2026-03-02 04:35:38.312 DEBUG [src.node.waku_node] Container stopped.
2026-03-02 04:35:38.312 DEBUG [src.node.waku_node] Stopping container with id 8237743a5f13
2026-03-02 04:35:38.529 ERROR [src.node.docker_mananger] Max retries reached for container 7a3a4772e97b. Exiting log stream.
2026-03-02 04:35:38.876 DEBUG [src.node.waku_node] Container stopped.
2026-03-02 04:35:38.877 DEBUG [src.node.waku_node] Stopping container with id 4dcc908fcff0
2026-03-02 04:35:39.126 ERROR [src.node.docker_mananger] Max retries reached for container 82c44f037a62. Exiting log stream.
2026-03-02 04:35:39.461 DEBUG [src.node.waku_node] Container stopped.
2026-03-02 04:35:39.462 DEBUG [src.node.waku_node] Stopping container with id d5c511e4fcee
2026-03-02 04:35:39.691 ERROR [src.node.docker_mananger] Max retries reached for container 8237743a5f13. Exiting log stream.
2026-03-02 04:35:39.995 DEBUG [src.node.waku_node] Container stopped.
2026-03-02 04:35:39.997 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-02 04:35:40.015 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-02 04:35:40.021 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-02 04:35:40.026 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-02 04:35:40.032 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-02 04:35:40.037 DEBUG [src.node.docker_mananger] No errors found in the waku logs.