logos-storage-nim/tools/storage-test/README.md

Storage Test Scripts

Helpers for running a local Logos Storage node and testing it against the Linode node.

Remote Node

The Linode node is running as a systemd service:

ssh storage@172.235.163.25
systemctl status logos-storage.service
journalctl -u logos-storage.service -f

Remote node ports:

Purpose	Address
P2P TCP	172.235.163.25:8070
Discovery UDP	172.235.163.25:8090
REST API	127.0.0.1:8080 on the Linode only

The REST API is not exposed publicly. Use the SSH tunnel managed by storage-test.sh.

Start A Local Node

Build the local binary first if needed:

make -j1 NIMFLAGS="-d:disableMarchNative"

Start a local node in the foreground:

tools/storage-test/start-local-node.sh

Start a libstorage-based local node in the foreground:

tools/storage-test/start-local-node.sh --client lib

Default local settings:

Setting	Default
Binary	./build/storage
Data dir	~/.logos/storage/local-node for storage, ~/.logos/storage/libstorage/node for lib
Log level	info
P2P TCP	8071
Discovery UDP	8091
REST API	127.0.0.1:8080
Lib IPC socket	~/.logos/storage/libstorage/storage_lib.sock
Network	logos.test

info is a good default log level: it shows startup, networking, and high-level node events without the volume of debug or trace.

Use debug when diagnosing behavior:

tools/storage-test/start-local-node.sh --log-level debug

Use trace only for detailed protocol/debug investigation because it can be noisy:

tools/storage-test/start-local-node.sh --log-level trace

Show all local-node options:

tools/storage-test/start-local-node.sh --help

The local node runs in the foreground. Press Ctrl-C to stop it.

Test Helper

Show commands:

tools/storage-test/storage-test.sh --help

Defaults:

Setting	Default
Remote SSH	storage@172.235.163.25
Remote API tunnel	127.0.0.1:18080
Local API	127.0.0.1:8080
Libstorage socket	~/.logos/storage/libstorage/storage_lib.sock
CID state file	~/.logos/storage/test/cids.log
Generated test files	~/.logos/storage/test/files/

cids.log is a simple upload history. Each upload appends the timestamp, target, returned CID, and source file path. It is useful when running frequent tests because you can recover old CIDs and delete them later without scrolling terminal history.

Recover the latest CID from the upload history:

tools/storage-test/storage-test.sh last-cid
tools/storage-test/storage-test.sh last-cid remote

SSH Tunnel

Start the tunnel:

tools/storage-test/storage-test.sh tunnel start

Check it:

tools/storage-test/storage-test.sh tunnel status

Stop it:

tools/storage-test/storage-test.sh tunnel stop

You do not have to stop the tunnel after each test. It is safe to leave it running while you are actively testing. Stop it when you are done, when you want to free local port 18080, or before changing tunnel settings.

Commands that target remote start the tunnel automatically if it is not already running.

Typical Workflow

Terminal 1: start local node and watch logs.

tools/storage-test/start-local-node.sh --log-level info

Terminal 2: upload random content to the Linode node.

CID="$(tools/storage-test/storage-test.sh remote upload-random 10M)"
printf '%s\n' "$CID"

If you prefer copy/paste, you can also run the upload command directly and copy the printed CID:

tools/storage-test/storage-test.sh remote upload-random 10M

Recover it later from the upload history:

CID="$(tools/storage-test/storage-test.sh last-cid remote)"

Ask the local node to fetch and store the content from the network:

tools/storage-test/storage-test.sh local fetch "$CID" --wait

Or stream the content through the local node without explicitly storing it first:

tools/storage-test/storage-test.sh local download "$CID" /tmp/logos-download.bin

Check local presence:

tools/storage-test/storage-test.sh local exists "$CID"

List local, remote, and lib CIDs:

tools/storage-test/storage-test.sh local list
tools/storage-test/storage-test.sh remote list
tools/storage-test/storage-test.sh lib list

Delete by CID:

tools/storage-test/storage-test.sh local delete "$CID"
tools/storage-test/storage-test.sh remote delete "$CID"

Delete all local CIDs:

tools/storage-test/storage-test.sh local delete-all --yes

Delete all remote CIDs:

tools/storage-test/storage-test.sh remote delete-all --yes

Libstorage Daemon Target

Build and start the libstorage daemon from tools/libstorage-cpp:

cd tools/libstorage-cpp
make
cd ../..
tools/storage-test/start-local-node.sh --client lib

Then use the lib target from the repository root:

tools/storage-test/storage-test.sh lib peerid
tools/storage-test/storage-test.sh lib upload README.md
tools/storage-test/storage-test.sh lib download <CID> /tmp/logos-lib-download.bin
tools/storage-test/storage-test.sh lib spr
tools/storage-test/storage-test.sh lib debug

Override the socket with STORAGE_LIB_SOCKET if the daemon was started with a non-default socket path.

Test Scenario

Run the first remote-to-local scenario against a standard local REST node:

tools/storage-test/storage-test.sh local test

Run the same scenario against the libstorage daemon:

tools/storage-test/storage-test.sh lib test

The scenario uploads random files to the remote Linode node, downloads them using the selected local target, validates SHA-256 hashes, and deletes involved CIDs from both sides. The default file sizes are 4K 1M 10M; override with TEST_FILE_SIZES.

The scenario prints a detailed progress summary and writes a Markdown report in the current directory:

./report-YYYY-MM-DD_HH-MM-SS.md

Useful API Endpoints

Operation	Endpoint
Upload	POST /api/storage/v1/data
List local content	GET /api/storage/v1/data
Delete local content	DELETE /api/storage/v1/data/{cid}
Fetch from network into node	POST /api/storage/v1/data/{cid}/network
Fetch progress	GET /api/storage/v1/data/{cid}/network/progress/{downloadId}
Stream from network	GET /api/storage/v1/data/{cid}/network/stream
Local existence check	GET /api/storage/v1/data/{cid}/exists
Storage space	GET /api/storage/v1/space

Cleanup

Stop the local node with Ctrl-C.

Stop the SSH tunnel when finished:

tools/storage-test/storage-test.sh tunnel stop

Remove local test data if desired:

rm -rf ~/.logos/storage/local-node
rm -rf ~/.logos/storage/test