mirror of https://github.com/status-im/migrate.git
Update documentation
This commit is contained in:
Matthias Kadenbach
parent
526c0a918e
commit
cd6e62049c
|
|
@ -10,7 +10,7 @@
|
|||
7. `make restore-import-paths` to restore import paths
|
||||
8. Push code and open Pull Request
|
||||
|
||||
Some more notes:
|
||||
Some more helpful commands:
|
||||
|
||||
* You can specify which database/ source tests to run:
|
||||
`make test-short SOURCE='file go-bindata' DATABASE='postgres cassandra'`
|
||||
|
|
|
|||
|
|
@ -0,0 +1,60 @@
|
|||
# FAQ
|
||||
|
||||
#### How is the code base structured?
|
||||
```
|
||||
/ package migrate (the heart of everything)
|
||||
/cli the CLI wrapper
|
||||
/database database driver and sub directories have the actual driver implementations
|
||||
/source source driver and sub directories have the actual driver implementations
|
||||
```
|
||||
|
||||
#### Why is there no `source/driver.go:Last()`?
|
||||
It's not needed. And unless the source has a "native" way to read a directory in reversed order,
|
||||
it might be expensive to do a full directory scan in order to get the last element.
|
||||
|
||||
#### What is a NilMigration? NilVersion?
|
||||
@TODO
|
||||
|
||||
#### What is the difference between uint(version) and int(targetVersion)?
|
||||
version refers to an existing migration version coming from a source and therefor can never be negative.
|
||||
targetVersion can either be a version OR represent a NilVersion, which equals -1.
|
||||
|
||||
#### What's the difference between Next/Previous and Up/Down?
|
||||
```
|
||||
1_first_migration.up next -> 2_second_migration.up ...
|
||||
1_first_migration.down <- previous 2_second_migration.down ...
|
||||
```
|
||||
|
||||
#### Why two separate files (up and down) for a migration?
|
||||
It makes all of our lives easier. No new markup/syntax to learn for users
|
||||
and existing database utility tools continue to work as expected.
|
||||
|
||||
#### How many migrations can migrate handle?
|
||||
Whatever the maximum positive signed integer value is for your platform.
|
||||
For 32bit it would be 2,147,483,647 migrations. Migrate only keeps references to
|
||||
the currently run and pre-fetched migrations in memory. Please note that some
|
||||
source drivers need to do build a full "directory" tree first, which puts some
|
||||
heat on the memory consumption.
|
||||
|
||||
#### Are the table tests in migrate_test.go bloated?
|
||||
Yes and no. There are duplicate test cases for sure but they don't hurt here. In fact
|
||||
the tests are very visual now and might help new users understand expected behaviors quickly.
|
||||
Migrate from version x to y and y is the last migration? Just check out the test for
|
||||
that particular case and know what's going on instantly.
|
||||
|
||||
#### What is Docker being used for?
|
||||
Only for testing. See [testing/docker.go](testing/docker.go)
|
||||
|
||||
#### Why not just use docker-compose?
|
||||
It doesn't give us enough runtime control for testing. We want to be able to bring up containers fast
|
||||
and whenever we want, not just once at the beginning of all tests.
|
||||
|
||||
#### Can I maintain my driver in my own repository?
|
||||
Yes, technically thats possible. We want to encourage you to contribute your driver to this respository though.
|
||||
The driver's functionality is dictated by migrate's interfaces. That means there should really
|
||||
just be one driver for a database/ source. We want to prevent a future where several drivers doing the exact same thing,
|
||||
just implemented a bit differently, co-exist somewhere on Github. If users have to do research first to find the
|
||||
"best" available driver for a database in order to get started, we would have failed as an open source community.
|
||||
|
||||
#### Can I mix multiple sources during a batch of migrations?
|
||||
No.
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
# Migrations
|
||||
|
||||
## Best practices: How to write migrations.
|
||||
|
||||
@TODO
|
||||
5
Makefile
5
Makefile
|
|
@ -106,9 +106,8 @@ release:
|
|||
|
||||
|
||||
define external_deps
|
||||
@echo -- $(1)
|
||||
@go list -f '{{join .Deps "\n"}}' $(1) | grep -v github.com/$(REPO_OWNER)/migrate | xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}'
|
||||
@#\n
|
||||
@echo '-- $(1)'; go list -f '{{join .Deps "\n"}}' $(1) | grep -v github.com/$(REPO_OWNER)/migrate | xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}'
|
||||
|
||||
endef
|
||||
|
||||
|
||||
|
|
|
|||
83
README.md
83
README.md
|
|
@ -1,17 +1,23 @@
|
|||
# migrate
|
||||
|
||||
[](https://travis-ci.org/mattes/migrate)
|
||||
[](https://godoc.org/github.com/mattes/migrate)
|
||||
[](https://coveralls.io/github/mattes/migrate?branch=v3.0-prev)
|
||||
[](https://packagecloud.io/mattes/migrate?filter=debs)
|
||||
|
||||
__Database migrations written in Go. Use as CLI or import as library.__
|
||||
# migrate
|
||||
|
||||
__Database migrations written in Go. Use as [CLI](#cli-usage) or import as [library](#use-in-your-go-project).__
|
||||
|
||||
* Migrate reads migrations from [sources](#migration-sources)
|
||||
and applies them in correct order to a [database](#databases).
|
||||
* Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof.
|
||||
(Keeps the drivers lightweight, too.)
|
||||
* Database drivers don't assume things or try to correct user input. When in doubt, fail.
|
||||
|
||||
|
||||
|
||||
## Databases
|
||||
|
||||
Database drivers are responsible for applying migrations to databases.
|
||||
Implementing a new database driver is easy. Just implement [database/driver interface](database/driver.go)
|
||||
Database drivers run migrations. [Add a new database?](database/driver.go)
|
||||
|
||||
* [PostgreSQL](database/postgres)
|
||||
* [Cassandra](database/cassandra)
|
||||
|
|
@ -24,10 +30,10 @@ Implementing a new database driver is easy. Just implement [database/driver inte
|
|||
* [Shell](database/shell)
|
||||
|
||||
|
||||
|
||||
## Migration Sources
|
||||
|
||||
Source Drivers read migrations from various locations. Implementing a new source driver
|
||||
is easy. Just implement the [source/driver interface](source/driver.go).
|
||||
Source drivers read migrations from local or remote sources. [Add a new source?](source/driver.go)
|
||||
|
||||
* [Filesystem](source/file) - read from fileystem (always included)
|
||||
* [Go-Bindata](source/go-bindata) - read from embedded binary data ([jteeuwen/go-bindata](https://github.com/jteeuwen/go-bindata))
|
||||
|
|
@ -36,56 +42,75 @@ is easy. Just implement the [source/driver interface](source/driver.go).
|
|||
* [Google Cloud Storage](source/google-cloud-storage) - read from Google Cloud Platform Storage
|
||||
|
||||
|
||||
|
||||
## CLI usage
|
||||
|
||||
__[CLI Documentation](cli/README.md)__
|
||||
* Simple wrapper around this library.
|
||||
* Handles ctrl+c (SIGINT) gracefully.
|
||||
* No config search paths, no config files, no magic ENV var injections.
|
||||
|
||||
|
||||
Example:
|
||||
__[CLI Documentation](cli)__
|
||||
|
||||
```
|
||||
go get -u -tags 'postgres' -o migrate github.com/mattes/migrate/cli
|
||||
|
||||
migrate -database postgres://localhost:5432/database up 2
|
||||
$ brew install migrate --with-postgres
|
||||
$ migrate -database postgres://localhost:5432/database up 2
|
||||
```
|
||||
|
||||
|
||||
## Use in your Go project
|
||||
|
||||
* API is stable and frozen for this release (v3.x).
|
||||
* Package migrate has no external dependencies.
|
||||
* Only import the drivers you need.
|
||||
(check [dependency_tree.txt](https://github.com/mattes/migrate/releases) for each driver)
|
||||
* To help prevent database corruptions, it supports graceful stops via `GracefulStop chan bool`.
|
||||
* Bring your own logger.
|
||||
* Uses `io.Reader` streams internally for low memory overhead.
|
||||
* Thread-safe.
|
||||
|
||||
__[Go Documentation](https://godoc.org/github.com/mattes/migrate)__
|
||||
|
||||
```go
|
||||
import (
|
||||
"github.com/mattes/migrate/migrate"
|
||||
_ "github.com/mattes/migrate/database/postgres"
|
||||
_ "github.com/mattes/migrate/source/github"
|
||||
"github.com/mattes/migrate/migrate"
|
||||
_ "github.com/mattes/migrate/database/postgres"
|
||||
_ "github.com/mattes/migrate/source/github"
|
||||
)
|
||||
|
||||
func main() {
|
||||
m, err := migrate.New("github://mattes:personal-access-token@mattes/migrate_test",
|
||||
"postgres://localhost:5432/database?sslmode=enable")
|
||||
m.Steps(2)
|
||||
m, err := migrate.New(
|
||||
"github://mattes:personal-access-token@mattes/migrate_test",
|
||||
"postgres://localhost:5432/database?sslmode=enable")
|
||||
m.Steps(2)
|
||||
}
|
||||
```
|
||||
|
||||
## Migration files
|
||||
|
||||
Each migration version has an up and down migration.
|
||||
Each migration has an up and down migration. [Why?](FAQ.md#why-two-separate-files-up-and-down-for-a-migration)
|
||||
|
||||
```
|
||||
1481574547_create_users_table.up.sql
|
||||
1481574547_create_users_table.down.sql
|
||||
```
|
||||
|
||||
## Development, Testing and Contributing
|
||||
|
||||
__[Guide](CONTRIBUTING.md)__
|
||||
[Best practices: How to write migrations.](MIGRATIONS.md)
|
||||
|
||||
|
||||
## Alternatives
|
||||
|
||||
* https://bitbucket.org/liamstask/goose
|
||||
* https://github.com/tanel/dbmigrate
|
||||
* https://github.com/BurntSushi/migration
|
||||
* https://github.com/DavidHuie/gomigrate
|
||||
* https://github.com/rubenv/sql-migrate
|
||||
## Development and Contributing
|
||||
|
||||
Yes, please! [`Makefile`](Makefile) is your friend,
|
||||
read the [development guide](CONTRIBUTING.md).
|
||||
|
||||
Also have a look at the [FAQ](FAQ.md).
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
__Alternatives__
|
||||
|
||||
https://bitbucket.org/liamstask/goose, https://github.com/tanel/dbmigrate,
|
||||
https://github.com/BurntSushi/migration, https://github.com/DavidHuie/gomigrate,
|
||||
https://github.com/rubenv/sql-migrate
|
||||
|
|
|
|||
|
|
@ -1,13 +1,36 @@
|
|||
# CLI
|
||||
# migrate CLI
|
||||
|
||||
## Installation
|
||||
|
||||
#### With Go toolchain
|
||||
|
||||
```
|
||||
# dowload, build and install the CLI tool
|
||||
# -tags takes database and source drivers and will only build those
|
||||
$ go get -u -tags 'postgres' -o migrate github.com/mattes/migrate/cli
|
||||
```
|
||||
|
||||
#### MacOS
|
||||
|
||||
```
|
||||
$ brew install migrate --with-postgres
|
||||
```
|
||||
|
||||
#### Linux (with deb package)
|
||||
|
||||
```
|
||||
# TODO: add key and repo
|
||||
$ apt-get update
|
||||
$ apt-get install migrate
|
||||
```
|
||||
|
||||
#### Download pre-build binary (Windows, MacOS, or Linux)
|
||||
|
||||
[Release Downloads](https://github.com/mattes/migrate/releases)
|
||||
|
||||
```
|
||||
$ curl -L https://github.com/mattes/migrate/releases/download/$version/migrate.$platform-amd64.tar.gz | tar xvz
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Usage
|
||||
|
||||
|
|
@ -31,13 +54,49 @@ Commands:
|
|||
down [N] Apply all or N down migrations
|
||||
drop Drop everyting inside database
|
||||
version Print current migration version
|
||||
|
||||
|
||||
# so let's say you want to run the first two migrations
|
||||
migrate -database postgres://localhost:5432/database up 2
|
||||
|
||||
# if your migrations are hosted on github
|
||||
migrate -source github://mattes:personal-access-token@mattes/migrate_test \
|
||||
-database postgres://localhost:5432/database down 2
|
||||
```
|
||||
|
||||
|
||||
So let's say you want to run the first two migrations
|
||||
|
||||
```
|
||||
$ migrate -database postgres://localhost:5432/database up 2
|
||||
```
|
||||
|
||||
If your migrations are hosted on github
|
||||
|
||||
```
|
||||
$ migrate -source github://mattes:personal-access-token@mattes/migrate_test \
|
||||
-database postgres://localhost:5432/database down 2
|
||||
```
|
||||
|
||||
The CLI will gracefully stop at a safe point when SIGINT (ctrl+c) is received.
|
||||
Send SIGKILL for immediate halt.
|
||||
|
||||
|
||||
|
||||
## Reading CLI arguments from somewhere else
|
||||
|
||||
##### ENV variables
|
||||
|
||||
```
|
||||
$ migrate -database "$MY_MIGRATE_DATABASE"
|
||||
```
|
||||
|
||||
##### JSON files
|
||||
|
||||
Check out https://stedolan.github.io/jq/
|
||||
|
||||
```
|
||||
$ migrate -database "$(cat config.json | jq '.database')"
|
||||
```
|
||||
|
||||
##### YAML files
|
||||
|
||||
````
|
||||
$ migrate -database "$(cat config/database.yml | ruby -ryaml -e "print YAML.load(STDIN.read)['database']")"
|
||||
$ migrate -database "$(cat config/database.yml | python -c 'import yaml,sys;print yaml.safe_load(sys.stdin)["database"]')"
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -20,13 +20,24 @@ const NilVersion int = -1
|
|||
var driversMu sync.RWMutex
|
||||
var drivers = make(map[string]Driver)
|
||||
|
||||
// Driver is an interface every driver must implement.
|
||||
// The driver implementation must pass the `Test` in database/testing.
|
||||
// Optionally provide a `WithInstance` function, so users can bypass `Open`
|
||||
// and use an existing database instance.
|
||||
// Driver is the interface every database driver must implement.
|
||||
//
|
||||
// Implementations must not assume things nor try to correct user input.
|
||||
// If in doubt, return an error.
|
||||
// How to implement a database driver?
|
||||
// 1. Implement this interface.
|
||||
// 2. Optionally, add a function named `WithInstance`.
|
||||
// This function should accept an existing DB instance and a Config{} struct
|
||||
// and return a driver instance.
|
||||
// 3. Add a test that calls database/testing.go:Test()
|
||||
// 4. Add own tests for Open(), WithInstance() (when provided) and Close().
|
||||
// All other functions are tested by tests in database/testing.
|
||||
// Saves you some time and makes sure all database drivers behave the same way.
|
||||
// 5. Call Register in init().
|
||||
//
|
||||
// Guidelines:
|
||||
// * Don't try to correct user input. Don't assume things.
|
||||
// When in doubt, return an error and explain the situation to the user.
|
||||
// * All configuration input must come from the URL string in func Open()
|
||||
// or the Config{} struct in WithInstance. Don't os.Getenv().
|
||||
type Driver interface {
|
||||
// Open returns a new driver instance configured with parameters
|
||||
// coming from the URL string. Migrate will call this function
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
mysql
|
||||
|
|
@ -14,10 +14,24 @@ import (
|
|||
var driversMu sync.RWMutex
|
||||
var drivers = make(map[string]Driver)
|
||||
|
||||
// Driver is an interface every driver must implement.
|
||||
// The driver implementation must pass the `Test` in source/testing.
|
||||
// Optionally provide a `WithInstance` function, so users can bypass `Open`
|
||||
// and use an existing source instance.
|
||||
// Driver is the interface every source driver must implement.
|
||||
//
|
||||
// How to implement a source driver?
|
||||
// 1. Implement this interface.
|
||||
// 2. Optionally, add a function named `WithInstance`.
|
||||
// This function should accept an existing source instance and a Config{} struct
|
||||
// and return a driver instance.
|
||||
// 3. Add a test that calls source/testing.go:Test()
|
||||
// 4. Add own tests for Open(), WithInstance() (when provided) and Close().
|
||||
// All other functions are tested by tests in source/testing.
|
||||
// Saves you some time and makes sure all source drivers behave the same way.
|
||||
// 5. Call Register in init().
|
||||
//
|
||||
// Guidelines:
|
||||
// * All configuration input must come from the URL string in func Open()
|
||||
// or the Config{} struct in WithInstance. Don't os.Getenv().
|
||||
// * Drivers are supposed to be read only.
|
||||
// * Ideally don't load any contents (into memory) in Open or WithInstance.
|
||||
type Driver interface {
|
||||
// Open returns a a new driver instance configured with parameters
|
||||
// coming from the URL string. Migrate will call this function
|
||||
|
|
|
|||
Loading…
Reference in New Issue