logos-messaging-nim/docs/tutorial/db-migration.md

Database Migration

This tutorial explains the database migration process in nim-waku.

Contributors Guide

Database Migration Flow

Nim-waku utilizes the built-in user_version variable that Sqlite provides for tracking the database versions. The user_version MUST be bumped up for every update on the database e.g, table schema/title change. Each update should be accompanied by a migration script to move the content of the old version of the database to the new version. The script MUST be added to the respective folder as explained in Migration Folder Structure with the proper naming as given in Migration Script Naming.

The migration is invoked whenever the database user_version is behind the target user_version indicated in the nim-waku application.
The respective migration scripts located in the migrations folder will be executed to upgrade the database from its old version to the target version.

Migration Folder Structure

The migrations folder is structured as below.

migrations/
├── message_store
│   ├── 00001_addMessageTable.up.sql
│   ├── 00002_addSenderTimeStamp.up.sql
│   ├── ...
└── peer_store
    └── 00001_addPeerTable.up.sql

The migration scripts are managed in two separate folders message_store and peer_store. The message_store folder contains the migration scripts related to the message store tables. Similarly, the peer_store folder contains the scripts relevant to the peer store tables.

Migration File Naming

The files in migrations folder MUST follow the following naming style in order to be properly included in the migration process. Files with invalid naming will be eliminated from the migration process.

<version number>_<migration script description>.<up|down>.sql

• version number: This number should match the target value of user_version.
• migration script description: A short description of what the migration script does.
• up|down: One of the keywords of up or down should be selected. up stands for upgrade and down means downgrade.

Example

A migration file with the name 00002_addTableX.up.sql should be interpreted as:

• 00002: The targeted user_version number.
• addTableX: What the script does.
• up: This script upgrades the database from user_version = 00001 to the user_version = 00002.

A downgrade migration file corresponding to the 00002_addTableX.up.sql can be e.g., 00001_removeTableX.down.sql and should be interpreted as:

• 00001: The targeted user_version number.
• removeTableX: What the script does.
• down: This script downgrades the database from user_version = 00002 to the user_version = 00001.

There can be more than one migration file for the same user_version. The migration process will consider all such files while upgrading/downgrading the database. Note that currently we DO NOT support down migration.

User Guide

Migrations work out of the box. However, if you want to be extra sure, please take a backup of the SQLite database prior to upgrading your nim-waku version since we currently don't support downgrades of DB.