190 lines
35 KiB
Plaintext

2026-01-02 04:22:28.484 DEBUG [tests.conftest] Running fixture setup: test_id
2026-01-02 04:22:28.485 DEBUG [tests.conftest] Running test: test_last_node_to_start_publishes with id: 2026-01-02_04-22-28__69a66988-8daa-4e34-a962-fd7f1d8d03a1
2026-01-02 04:22:28.485 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-01-02 04:22:28.485 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-01-02 04:22:28.485 DEBUG [src.steps.relay] Running fixture setup: setup_main_relay_nodes
2026-01-02 04:22:28.492 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-02 04:22:28.492 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-01-02_04-22-28__69a66988-8daa-4e34-a962-fd7f1d8d03a1__wakuorg_nwaku:latest.log
2026-01-02 04:22:28.492 DEBUG [src.node.waku_node] Starting Node...
2026-01-02 04:22:28.492 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-02 04:22:28.494 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-02 04:22:28.494 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.234.95
2026-01-02 04:22:28.494 DEBUG [src.node.docker_mananger] Generated ports ['35099', '35100', '35101', '35102', '35103']
2026-01-02 04:22:28.494 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-02 04:22:28.494 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-02 04:22:28.494 DEBUG [src.node.waku_node] Using volumes []
2026-01-02 04:22:28.495 DEBUG [src.node.docker_mananger] docker run -i -t -p 35099:35099 -p 35100:35100 -p 35101:35101 -p 35102:35102 -p 35103:35103 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=35101 --rest-port=35099 --tcp-port=35100 --discv5-udp-port=35102 --rest-address=0.0.0.0 --nat=extip:172.18.234.95 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=dcba1bbc97cbdbde4ada1b5b61419fa4d44a3d45b0cf2b3eddabafe399c71d38 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=35103 --metrics-logging=true --relay=true
2026-01-02 04:22:28.737 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.234.95 waku 3cea9d4469045c12ac39479dcbf505ca6cd542b26c7b99df4b09916357ae8e22
2026-01-02 04:22:28.778 DEBUG [src.node.docker_mananger] Container started with ID 3cea9d446904. Setting up logs at ./log/docker/node1_2026-01-02_04-22-28__69a66988-8daa-4e34-a962-fd7f1d8d03a1__wakuorg_nwaku:latest.log
2026-01-02 04:22:28.780 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 35099
2026-01-02 04:22:28.780 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-02 04:22:29.780 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35099/health" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:29.784 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-02 04:22:29.784 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-02 04:22:29.784 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35099/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:29.786 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.234.95/tcp/35100/p2p/16Uiu2HAmKXNQiejEAvfkKbc1iFWBwTaRbdLE8FgJoP4urYkFRVhD","/ip4/172.18.234.95/tcp/35101/ws/p2p/16Uiu2HAmKXNQiejEAvfkKbc1iFWBwTaRbdLE8FgJoP4urYkFRVhD"],"enrUri":"enr:-L24QMRyLLY1nXZDCzIHCjmYoYDnVz8wVmalxeL4pUFJXUHZf8mAqi6udFTx7fa9i4nFEhso_ukeOCFPNfnyXNgE1p0CgmlkgnY0gmlwhKwS6l-KbXVsdGlhZGRyc5YACASsEupfBokcAAoErBLqXwaJHd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDZg4atgl4jadlgOh9H1v6mi2qynKfo93RFZJIBtjOg8yDdGNwgokcg3VkcIKJHoV3YWt1MgE"}'
2026-01-02 04:22:29.786 INFO [src.node.waku_node] REST service is ready !!
2026-01-02 04:22:29.792 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-02 04:22:29.793 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-01-02_04-22-28__69a66988-8daa-4e34-a962-fd7f1d8d03a1__wakuorg_nwaku:latest.log
2026-01-02 04:22:29.793 DEBUG [src.node.waku_node] Starting Node...
2026-01-02 04:22:29.793 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-02 04:22:29.794 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-02 04:22:29.794 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.84.129
2026-01-02 04:22:29.794 DEBUG [src.node.docker_mananger] Generated ports ['7236', '7237', '7238', '7239', '7240']
2026-01-02 04:22:29.795 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-02 04:22:29.795 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-02 04:22:29.795 DEBUG [src.node.waku_node] Using volumes []
2026-01-02 04:22:29.795 DEBUG [src.node.docker_mananger] docker run -i -t -p 7236:7236 -p 7237:7237 -p 7238:7238 -p 7239:7239 -p 7240:7240 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=7238 --rest-port=7236 --tcp-port=7237 --discv5-udp-port=7239 --rest-address=0.0.0.0 --nat=extip:172.18.84.129 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=8cb1eed7df3fb15a3abb3acfaf7176e4ac49e106c54a512af8ddaf67df7e2cce --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=7240 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QMRyLLY1nXZDCzIHCjmYoYDnVz8wVmalxeL4pUFJXUHZf8mAqi6udFTx7fa9i4nFEhso_ukeOCFPNfnyXNgE1p0CgmlkgnY0gmlwhKwS6l-KbXVsdGlhZGRyc5YACASsEupfBokcAAoErBLqXwaJHd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDZg4atgl4jadlgOh9H1v6mi2qynKfo93RFZJIBtjOg8yDdGNwgokcg3VkcIKJHoV3YWt1MgE
2026-01-02 04:22:29.998 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.84.129 waku 617387d50770ce409898e7201aec9d956e68674403a50600b51c4f6181b71869
2026-01-02 04:22:30.031 DEBUG [src.node.docker_mananger] Container started with ID 617387d50770. Setting up logs at ./log/docker/node2_2026-01-02_04-22-28__69a66988-8daa-4e34-a962-fd7f1d8d03a1__wakuorg_nwaku:latest.log
2026-01-02 04:22:30.032 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 7236
2026-01-02 04:22:30.032 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-02 04:22:31.032 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7236/health" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:31.050 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-02 04:22:31.050 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-02 04:22:31.050 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7236/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:31.056 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.84.129/tcp/7237/p2p/16Uiu2HAmH9L5MWgdWu33hxYwsKRTF5FSvJvNdHD1QbcpEC51L7Ax","/ip4/172.18.84.129/tcp/7238/ws/p2p/16Uiu2HAmH9L5MWgdWu33hxYwsKRTF5FSvJvNdHD1QbcpEC51L7Ax"],"enrUri":"enr:-L24QBJ36Ckne4vdXj-XyovrEjqukPbcC5MrIKSNshDjQv5neAmYtilXETPt4ie6p0rH-Q3Qy5PlKso-XSSs6b0ce1UCgmlkgnY0gmlwhKwSVIGKbXVsdGlhZGRyc5YACASsElSBBhxFAAoErBJUgQYcRt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDQrFQU2ZTyfPudzMTjv-jaPqynr8ZUhG5pnnlyV2rItGDdGNwghxFg3VkcIIcR4V3YWt1MgE"}'
2026-01-02 04:22:31.056 INFO [src.node.waku_node] REST service is ready !!
2026-01-02 04:22:31.057 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7236/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.234.95/tcp/35100/p2p/16Uiu2HAmKXNQiejEAvfkKbc1iFWBwTaRbdLE8FgJoP4urYkFRVhD"]'
2026-01-02 04:22:31.061 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:22:31.062 DEBUG [src.steps.relay] Running fixture setup: setup_optional_relay_nodes
2026-01-02 04:22:31.069 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-02 04:22:31.069 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-01-02_04-22-28__69a66988-8daa-4e34-a962-fd7f1d8d03a1__wakuorg_nwaku:latest.log
2026-01-02 04:22:31.069 DEBUG [src.node.waku_node] Starting Node...
2026-01-02 04:22:31.069 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-02 04:22:31.071 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-02 04:22:31.071 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.218.181
2026-01-02 04:22:31.071 DEBUG [src.node.docker_mananger] Generated ports ['21793', '21794', '21795', '21796', '21797']
2026-01-02 04:22:31.071 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-02 04:22:31.071 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-02 04:22:31.072 DEBUG [src.node.waku_node] Using volumes []
2026-01-02 04:22:31.072 DEBUG [src.node.docker_mananger] docker run -i -t -p 21793:21793 -p 21794:21794 -p 21795:21795 -p 21796:21796 -p 21797:21797 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=21795 --rest-port=21793 --tcp-port=21794 --discv5-udp-port=21796 --rest-address=0.0.0.0 --nat=extip:172.18.218.181 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=7bc47d033aeb4ee8e70af253a36decf191df10dd98f6ecebaa8c8ad75bdcce3e --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=21797 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QMRyLLY1nXZDCzIHCjmYoYDnVz8wVmalxeL4pUFJXUHZf8mAqi6udFTx7fa9i4nFEhso_ukeOCFPNfnyXNgE1p0CgmlkgnY0gmlwhKwS6l-KbXVsdGlhZGRyc5YACASsEupfBokcAAoErBLqXwaJHd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDZg4atgl4jadlgOh9H1v6mi2qynKfo93RFZJIBtjOg8yDdGNwgokcg3VkcIKJHoV3YWt1MgE
2026-01-02 04:22:31.255 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.218.181 waku a685918865a990769dd746813d51bf7ab23aef38c422091d18b2339127d632af
2026-01-02 04:22:31.285 DEBUG [src.node.docker_mananger] Container started with ID a685918865a9. Setting up logs at ./log/docker/node3_2026-01-02_04-22-28__69a66988-8daa-4e34-a962-fd7f1d8d03a1__wakuorg_nwaku:latest.log
2026-01-02 04:22:31.286 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 21793
2026-01-02 04:22:31.286 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-02 04:22:32.286 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:21793/health" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:32.296 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-02 04:22:32.296 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-02 04:22:32.297 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:21793/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:32.301 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.218.181/tcp/21794/p2p/16Uiu2HAmMj2LZL41DTCixbnZtCQtn8Le8nAGqFQH5ViFZSWa4jWN","/ip4/172.18.218.181/tcp/21795/ws/p2p/16Uiu2HAmMj2LZL41DTCixbnZtCQtn8Le8nAGqFQH5ViFZSWa4jWN"],"enrUri":"enr:-L24QE16BepG3gzVTFz4dsqnwtsTqnm7YZg0MRs1OP4k1BoNZQXrz997mV81sj9vlEUy1w_OG-onVq15vhGLhjYfuwgCgmlkgnY0gmlwhKwS2rWKbXVsdGlhZGRyc5YACASsEtq1BlUiAAoErBLatQZVI90DgnJzhQADAQAAiXNlY3AyNTZrMaEDhsHAsgyWPVpeFiutArpsoLG-HCYVgHKDGG2u-mZHU3eDdGNwglUig3VkcIJVJIV3YWt1MgE"}'
2026-01-02 04:22:32.301 INFO [src.node.waku_node] REST service is ready !!
2026-01-02 04:22:32.301 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:21793/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.234.95/tcp/35100/p2p/16Uiu2HAmKXNQiejEAvfkKbc1iFWBwTaRbdLE8FgJoP4urYkFRVhD"]'
2026-01-02 04:22:32.306 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:22:32.313 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-02 04:22:32.313 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-01-02_04-22-28__69a66988-8daa-4e34-a962-fd7f1d8d03a1__wakuorg_nwaku:latest.log
2026-01-02 04:22:32.313 DEBUG [src.node.waku_node] Starting Node...
2026-01-02 04:22:32.313 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-02 04:22:32.315 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-02 04:22:32.315 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.53.49
2026-01-02 04:22:32.315 DEBUG [src.node.docker_mananger] Generated ports ['7683', '7684', '7685', '7686', '7687']
2026-01-02 04:22:32.315 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-02 04:22:32.315 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-02 04:22:32.316 DEBUG [src.node.waku_node] Using volumes []
2026-01-02 04:22:32.316 DEBUG [src.node.docker_mananger] docker run -i -t -p 7683:7683 -p 7684:7684 -p 7685:7685 -p 7686:7686 -p 7687:7687 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=7685 --rest-port=7683 --tcp-port=7684 --discv5-udp-port=7686 --rest-address=0.0.0.0 --nat=extip:172.18.53.49 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=11d770b7ce8cdff8200be5bcbd8c0a781d5b8cd124c6d5ba2deaa5077cbd315f --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=7687 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QMRyLLY1nXZDCzIHCjmYoYDnVz8wVmalxeL4pUFJXUHZf8mAqi6udFTx7fa9i4nFEhso_ukeOCFPNfnyXNgE1p0CgmlkgnY0gmlwhKwS6l-KbXVsdGlhZGRyc5YACASsEupfBokcAAoErBLqXwaJHd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDZg4atgl4jadlgOh9H1v6mi2qynKfo93RFZJIBtjOg8yDdGNwgokcg3VkcIKJHoV3YWt1MgE
2026-01-02 04:22:32.510 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.53.49 waku 2bd0afc83fb0e1f62293a53f19b958a3f1a1fc338290440362c8347fb2c4a2de
2026-01-02 04:22:32.542 DEBUG [src.node.docker_mananger] Container started with ID 2bd0afc83fb0. Setting up logs at ./log/docker/node4_2026-01-02_04-22-28__69a66988-8daa-4e34-a962-fd7f1d8d03a1__wakuorg_nwaku:latest.log
2026-01-02 04:22:32.543 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 7683
2026-01-02 04:22:32.543 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-02 04:22:33.543 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7683/health" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:33.552 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"READY"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-02 04:22:33.553 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-02 04:22:33.554 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7683/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:33.557 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.53.49/tcp/7684/p2p/16Uiu2HAmE7LyVtkDrEnXZF9zCFHoA2haNN5PPJw1QcX6giKq6bEN","/ip4/172.18.53.49/tcp/7685/ws/p2p/16Uiu2HAmE7LyVtkDrEnXZF9zCFHoA2haNN5PPJw1QcX6giKq6bEN"],"enrUri":"enr:-L24QGgLL-fv6K-qn5EGYTju1OE5YjwRYa8yvOMFzroAqu3NEjfRsB_v2A6YFrTTx4UF9FKsE_ypFE1Em6hrpVH2vgcCgmlkgnY0gmlwhKwSNTGKbXVsdGlhZGRyc5YACASsEjUxBh4EAAoErBI1MQYeBd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDFZwboREzj2dRxZy7Y0MJ09w545v-FhPZUwCisvFra5eDdGNwgh4Eg3VkcIIeBoV3YWt1MgE"}'
2026-01-02 04:22:33.557 INFO [src.node.waku_node] REST service is ready !!
2026-01-02 04:22:33.558 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7683/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.234.95/tcp/35100/p2p/16Uiu2HAmKXNQiejEAvfkKbc1iFWBwTaRbdLE8FgJoP4urYkFRVhD"]'
2026-01-02 04:22:33.560 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:22:33.566 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-02 04:22:33.567 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-01-02_04-22-28__69a66988-8daa-4e34-a962-fd7f1d8d03a1__wakuorg_nwaku:latest.log
2026-01-02 04:22:33.567 DEBUG [src.node.waku_node] Starting Node...
2026-01-02 04:22:33.567 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-02 04:22:33.568 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-02 04:22:33.568 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.9.131
2026-01-02 04:22:33.568 DEBUG [src.node.docker_mananger] Generated ports ['56017', '56018', '56019', '56020', '56021']
2026-01-02 04:22:33.568 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-02 04:22:33.569 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-02 04:22:33.569 DEBUG [src.node.waku_node] Using volumes []
2026-01-02 04:22:33.569 DEBUG [src.node.docker_mananger] docker run -i -t -p 56017:56017 -p 56018:56018 -p 56019:56019 -p 56020:56020 -p 56021:56021 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=56019 --rest-port=56017 --tcp-port=56018 --discv5-udp-port=56020 --rest-address=0.0.0.0 --nat=extip:172.18.9.131 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=df0defcb12a05ffa7056d0c9bbeb5a2fad7ebaac1adba975ceaac096fc2dedfd --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=56021 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QMRyLLY1nXZDCzIHCjmYoYDnVz8wVmalxeL4pUFJXUHZf8mAqi6udFTx7fa9i4nFEhso_ukeOCFPNfnyXNgE1p0CgmlkgnY0gmlwhKwS6l-KbXVsdGlhZGRyc5YACASsEupfBokcAAoErBLqXwaJHd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDZg4atgl4jadlgOh9H1v6mi2qynKfo93RFZJIBtjOg8yDdGNwgokcg3VkcIKJHoV3YWt1MgE
2026-01-02 04:22:33.758 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.9.131 waku d36db9cf691521de48c0a66991c906613e2c25a43c0cc178026f8cb376ec5b7f
2026-01-02 04:22:33.789 DEBUG [src.node.docker_mananger] Container started with ID d36db9cf6915. Setting up logs at ./log/docker/node5_2026-01-02_04-22-28__69a66988-8daa-4e34-a962-fd7f1d8d03a1__wakuorg_nwaku:latest.log
2026-01-02 04:22:33.789 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 56017
2026-01-02 04:22:33.789 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-02 04:22:34.790 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56017/health" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:34.796 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-02 04:22:34.798 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-02 04:22:34.799 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56017/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:34.804 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.9.131/tcp/56018/p2p/16Uiu2HAkyqZCAFoyA8PCGMEvxw5SCqBXaJ8iJVeGBDjQfWwVSK5e","/ip4/172.18.9.131/tcp/56019/ws/p2p/16Uiu2HAkyqZCAFoyA8PCGMEvxw5SCqBXaJ8iJVeGBDjQfWwVSK5e"],"enrUri":"enr:-L24QHSMUa8As0O2YYgilxVmm_2y0vQTvlbyUtFlvHwYX2NOUyYhpjCZyia9wuqlyaWGR9hw28UXafVLrIB9vF3q0xUCgmlkgnY0gmlwhKwSCYOKbXVsdGlhZGRyc5YACASsEgmDBtrSAAoErBIJgwba090DgnJzhQADAQAAiXNlY3AyNTZrMaECQY0DZuWD5H6zu2SwuOuc19kGzuD71lekyw3J1vq9iD2DdGNwgtrSg3VkcILa1IV3YWt1MgE"}'
2026-01-02 04:22:34.805 INFO [src.node.waku_node] REST service is ready !!
2026-01-02 04:22:34.806 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56017/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.234.95/tcp/35100/p2p/16Uiu2HAmKXNQiejEAvfkKbc1iFWBwTaRbdLE8FgJoP4urYkFRVhD"]'
2026-01-02 04:22:34.809 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:22:34.810 DEBUG [src.steps.relay] Running fixture setup: subscribe_main_relay_nodes
2026-01-02 04:22:34.810 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35099/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-02 04:22:34.817 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:22:34.817 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7236/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-02 04:22:34.823 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:22:34.823 DEBUG [src.steps.relay] Running fixture setup: subscribe_optional_relay_nodes
2026-01-02 04:22:34.824 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:21793/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-02 04:22:34.829 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:22:34.830 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7683/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-02 04:22:34.836 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:22:34.837 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56017/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-02 04:22:34.844 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:22:34.844 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:35099/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-01-02 04:22:34.854 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:22:34.856 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-01-02 04:22:34.956 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-01-02 04:22:34.956 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35099/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:34.959 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767327754844530432,"ephemeral":false,"proof":""}]'
2026-01-02 04:22:34.960 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-01-02 04:22:34.960 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7236/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:34.963 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767327754844530432,"ephemeral":false,"proof":""}]'
2026-01-02 04:22:34.964 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-01-02 04:22:34.964 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:21793/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:34.966 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767327754844530432,"ephemeral":false,"proof":""}]'
2026-01-02 04:22:34.968 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-01-02 04:22:34.968 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7683/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:34.970 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767327754844530432,"ephemeral":false,"proof":""}]'
2026-01-02 04:22:34.971 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-01-02 04:22:34.971 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56017/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:34.974 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767327754844530432,"ephemeral":false,"proof":""}]'
2026-01-02 04:22:34.975 INFO [src.steps.relay] WARM UP successful!!
2026-01-02 04:22:34.976 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:56017/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-01-02 04:22:34.980 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-02 04:22:34.980 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-01-02 04:22:35.081 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-01-02 04:22:35.081 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:35099/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:35.084 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767327754976542632,"ephemeral":false,"proof":""}]'
2026-01-02 04:22:35.086 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-01-02 04:22:35.086 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7236/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:35.088 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767327754976542632,"ephemeral":false,"proof":""}]'
2026-01-02 04:22:35.089 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-01-02 04:22:35.090 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:21793/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:35.092 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767327754976542632,"ephemeral":false,"proof":""}]'
2026-01-02 04:22:35.093 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-01-02 04:22:35.093 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7683/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:35.095 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767327754976542632,"ephemeral":false,"proof":""}]'
2026-01-02 04:22:35.097 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-01-02 04:22:35.097 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56017/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-02 04:22:35.099 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1767327754976542632,"ephemeral":false,"proof":""}]'
2026-01-02 04:22:35.103 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-01-02 04:22:35.104 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-01-02 04:22:35.104 DEBUG [src.node.waku_node] Stopping container with id 3cea9d446904
2026-01-02 04:22:35.665 DEBUG [src.node.waku_node] Container stopped.
2026-01-02 04:22:35.665 DEBUG [src.node.waku_node] Stopping container with id 617387d50770
2026-01-02 04:22:36.191 DEBUG [src.node.waku_node] Container stopped.
2026-01-02 04:22:36.192 DEBUG [src.node.waku_node] Stopping container with id a685918865a9
2026-01-02 04:22:36.532 ERROR [src.node.docker_mananger] Max retries reached for container 3cea9d446904. Exiting log stream.
2026-01-02 04:22:36.722 DEBUG [src.node.waku_node] Container stopped.
2026-01-02 04:22:36.722 DEBUG [src.node.waku_node] Stopping container with id 2bd0afc83fb0
2026-01-02 04:22:37.004 ERROR [src.node.docker_mananger] Max retries reached for container 617387d50770. Exiting log stream.
2026-01-02 04:22:37.230 DEBUG [src.node.waku_node] Container stopped.
2026-01-02 04:22:37.231 DEBUG [src.node.waku_node] Stopping container with id d36db9cf6915
2026-01-02 04:22:37.535 ERROR [src.node.docker_mananger] Max retries reached for container a685918865a9. Exiting log stream.
2026-01-02 04:22:37.722 DEBUG [src.node.waku_node] Container stopped.
2026-01-02 04:22:37.724 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-01-02 04:22:37.753 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-02 04:22:37.760 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-02 04:22:37.766 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-02 04:22:37.772 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-02 04:22:37.779 DEBUG [src.node.docker_mananger] No errors found in the waku logs.