diff --git a/README.md b/README.md index 0b38a5e..307fe9b 100644 --- a/README.md +++ b/README.md @@ -6,16 +6,15 @@ ![Supported Go Versions](https://img.shields.io/badge/Go-1.11%2C%201.12-lightgrey.svg) [![GitHub Release](https://img.shields.io/github/release/golang-migrate/migrate.svg)](https://github.com/golang-migrate/migrate/releases) - # migrate __Database migrations written in Go. Use as [CLI](#cli-usage) or import as [library](#use-in-your-go-project).__ - * Migrate reads migrations from [sources](#migration-sources) +* Migrate reads migrations from [sources](#migration-sources) and applies them in correct order to a [database](#databases). - * Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof. +* Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof. (Keeps the drivers lightweight, too.) - * Database drivers don't assume things or try to correct user input. When in doubt, fail. +* Database drivers don't assume things or try to correct user input. When in doubt, fail. Forked from [mattes/migrate](https://github.com/mattes/migrate) @@ -23,21 +22,21 @@ Forked from [mattes/migrate](https://github.com/mattes/migrate) Database drivers run migrations. [Add a new database?](database/driver.go) - * [PostgreSQL](database/postgres) - * [Redshift](database/redshift) - * [Ql](database/ql) - * [Cassandra](database/cassandra) - * [SQLite](database/sqlite3) ([todo #165](https://github.com/mattes/migrate/issues/165)) - * [MySQL/ MariaDB](database/mysql) - * [Neo4j](database/neo4j) ([todo #167](https://github.com/mattes/migrate/issues/167)) - * [MongoDB](database/mongodb) - * [CrateDB](database/crate) ([todo #170](https://github.com/mattes/migrate/issues/170)) - * [Shell](database/shell) ([todo #171](https://github.com/mattes/migrate/issues/171)) - * [Google Cloud Spanner](database/spanner) - * [CockroachDB](database/cockroachdb) - * [ClickHouse](database/clickhouse) - * [Firebird](database/firebird) ([todo #49](https://github.com/golang-migrate/migrate/issues/49)) - * [MS SQL Server](database/sqlserver) +* [PostgreSQL](database/postgres) +* [Redshift](database/redshift) +* [Ql](database/ql) +* [Cassandra](database/cassandra) +* [SQLite](database/sqlite3) ([todo #165](https://github.com/mattes/migrate/issues/165)) +* [MySQL/ MariaDB](database/mysql) +* [Neo4j](database/neo4j) ([todo #167](https://github.com/mattes/migrate/issues/167)) +* [MongoDB](database/mongodb) +* [CrateDB](database/crate) ([todo #170](https://github.com/mattes/migrate/issues/170)) +* [Shell](database/shell) ([todo #171](https://github.com/mattes/migrate/issues/171)) +* [Google Cloud Spanner](database/spanner) +* [CockroachDB](database/cockroachdb) +* [ClickHouse](database/clickhouse) +* [Firebird](database/firebird) ([todo #49](https://github.com/golang-migrate/migrate/issues/49)) +* [MS SQL Server](database/sqlserver) ### Database URLs @@ -49,6 +48,7 @@ Explicitly, the following characters need to be escaped: `!`, `#`, `$`, `%`, `&`, `'`, `(`, `)`, `*`, `+`, `,`, `/`, `:`, `;`, `=`, `?`, `@`, `[`, `]` It's easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below: + ```bash $ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))' String to encode: FAKEpassword!#$%&'()*+,/:;=?@[] @@ -63,44 +63,42 @@ $ Source drivers read migrations from local or remote sources. [Add a new source?](source/driver.go) - * [Filesystem](source/file) - read from fileystem - * [Go-Bindata](source/go_bindata) - read from embedded binary data ([jteeuwen/go-bindata](https://github.com/jteeuwen/go-bindata)) - * [Github](source/github) - read from remote Github repositories - * [Gitlab](source/gitlab) - read from remote Gitlab repositories - * [AWS S3](source/aws_s3) - read from Amazon Web Services S3 - * [Google Cloud Storage](source/google_cloud_storage) - read from Google Cloud Platform Storage - - +* [Filesystem](source/file) - read from filesystem +* [Go-Bindata](source/go_bindata) - read from embedded binary data ([jteeuwen/go-bindata](https://github.com/jteeuwen/go-bindata)) +* [Github](source/github) - read from remote Github repositories +* [Gitlab](source/gitlab) - read from remote Gitlab repositories +* [AWS S3](source/aws_s3) - read from Amazon Web Services S3 +* [Google Cloud Storage](source/google_cloud_storage) - read from Google Cloud Platform Storage ## CLI usage - * Simple wrapper around this library. - * Handles ctrl+c (SIGINT) gracefully. - * No config search paths, no config files, no magic ENV var injections. +* Simple wrapper around this library. +* Handles ctrl+c (SIGINT) gracefully. +* No config search paths, no config files, no magic ENV var injections. -__[CLI Documentation](cli)__ +__[CLI Documentation](cmd/migrate)__ -### Basic usage: +### Basic usage -``` +```bash $ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2 ``` ### Docker usage -``` +```bash $ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate -path=/migrations/ -database postgres://localhost:5432/database up 2 ``` ## Use in your Go project - * API is stable and frozen for this release (v3 & v4). - * Uses [Go modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) to manage dependencies. - * To help prevent database corruptions, it supports graceful stops via `GracefulStop chan bool`. - * Bring your own logger. - * Uses `io.Reader` streams internally for low memory overhead. - * Thread-safe and no goroutine leaks. +* API is stable and frozen for this release (v3 & v4). +* Uses [Go modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) to manage dependencies. +* To help prevent database corruptions, it supports graceful stops via `GracefulStop chan bool`. +* Bring your own logger. +* Uses `io.Reader` streams internally for low memory overhead. +* Thread-safe and no goroutine leaks. __[Go Documentation](https://godoc.org/github.com/golang-migrate/migrate)__ @@ -144,7 +142,7 @@ func main() { Each migration has an up and down migration. [Why?](FAQ.md#why-two-separate-files-up-and-down-for-a-migration) -``` +```bash 1481574547_create_users_table.up.sql 1481574547_create_users_table.down.sql ``` @@ -166,8 +164,6 @@ read the [development guide](CONTRIBUTING.md). Also have a look at the [FAQ](FAQ.md). - - --- Looking for alternatives? [https://awesome-go.com/#database](https://awesome-go.com/#database). diff --git a/cli/README.md b/cli/README.md index b426521..cffeb29 100644 --- a/cli/README.md +++ b/cli/README.md @@ -1,125 +1,3 @@ -# migrate CLI +# Deprecated -## Installation - -#### Download pre-built binary (Windows, MacOS, or Linux) - -[Release Downloads](https://github.com/golang-migrate/migrate/releases) - -``` -$ curl -L https://github.com/golang-migrate/migrate/releases/download/$version/migrate.$platform-amd64.tar.gz | tar xvz -``` - -#### MacOS - -``` -$ brew install golang-migrate -``` - -#### Linux (*.deb package) - -``` -$ curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey | apt-key add - -$ echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/migrate.list -$ apt-get update -$ apt-get install -y migrate -``` - -#### With Go toolchain - -##### Unversioned -``` -$ go get -tags 'postgres' -u github.com/golang-migrate/migrate/cmd/migrate -``` - -##### Versioned - -``` -$ go get -u -d github.com/golang-migrate/migrate/cmd/migrate -$ cd $GOPATH/src/github.com/golang-migrate/migrate/cmd/migrate -$ git checkout $TAG # e.g. v4.1.0 -$ go build -tags 'postgres' -ldflags="-X main.Version=$(git describe --tags)" -o $GOPATH/bin/migrate github.com/golang-migrate/migrate/cmd/migrate -``` - -##### Notes: -1. Requires a version of Go that [supports modules](https://golang.org/cmd/go/#hdr-Preliminary_module_support). e.g. Go 1.11+ -1. This example builds the cli which will only work with postgres. In order -to build the cli for use with other databases, replace the `postgres` build tag -with the appropriate database tag(s) for the databases desired. The tags -correspond to the names of the sub-packages underneath the -[`database`](../database) package. -1. Similarly to the database build tags, if you need to support other sources, use the appropriate build tag(s). -1. Support for build constraints will be removed in the future: https://github.com/golang-migrate/migrate/issues/60 - - -## Usage - -``` -$ migrate -help -Usage: migrate OPTIONS COMMAND [arg...] - migrate [ -version | -help ] - -Options: - -source Location of the migrations (driver://url) - -path Shorthand for -source=file://path - -database Run migrations against this database (driver://url) - -prefetch N Number of migrations to load in advance before executing (default 10) - -lock-timeout N Allow N seconds to acquire database lock (default 15) - -verbose Print verbose logging - -version Print version - -help Print usage - -Commands: - create [-ext E] [-dir D] [-seq] [-digits N] [-format] NAME - Create a set of timestamped up/down migrations titled NAME, in directory D with extension E. - Use -seq option to generate sequential up/down migrations with N digits. - Use -format option to specify a Go time format string. - goto V Migrate to version V - up [N] Apply all or N up migrations - down [N] Apply all or N down migrations - drop Drop everyting inside database - force V Set version V but don't run migration (ignores dirty state) - version Print current migration version -``` - - -So let's say you want to run the first two migrations - -``` -$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2 -``` - -If your migrations are hosted on github - -``` -$ migrate -source github://mattes:personal-access-token@mattes/migrate_test \ - -database postgres://localhost:5432/database down 2 -``` - -The CLI will gracefully stop at a safe point when SIGINT (ctrl+c) is received. -Send SIGKILL for immediate halt. - - - -## Reading CLI arguments from somewhere else - -##### ENV variables - -``` -$ migrate -database "$MY_MIGRATE_DATABASE" -``` - -##### JSON files - -Check out https://stedolan.github.io/jq/ - -``` -$ migrate -database "$(cat config.json | jq '.database')" -``` - -##### YAML files - -``` -$ migrate -database "$(cat config/database.yml | ruby -ryaml -e "print YAML.load(STDIN.read)['database']")" -$ migrate -database "$(cat config/database.yml | python -c 'import yaml,sys;print yaml.safe_load(sys.stdin)["database"]')" -``` +Use [cmd/migrate](../cmd/migrate) instead diff --git a/cli/examples/Dockerfile b/cli/examples/Dockerfile deleted file mode 100644 index f6cd90c..0000000 --- a/cli/examples/Dockerfile +++ /dev/null @@ -1,12 +0,0 @@ -FROM ubuntu:xenial - -RUN apt-get update && \ - apt-get install -y curl apt-transport-https - -RUN curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey | apt-key add - && \ - echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ xenial main" > /etc/apt/sources.list.d/migrate.list && \ - apt-get update && \ - apt-get install -y migrate - -RUN migrate -version - diff --git a/cmd/migrate/README.md b/cmd/migrate/README.md new file mode 100644 index 0000000..6c78369 --- /dev/null +++ b/cmd/migrate/README.md @@ -0,0 +1,123 @@ +# migrate CLI + +## Installation + +### Download pre-built binary (Windows, MacOS, or Linux) + +[Release Downloads](https://github.com/golang-migrate/migrate/releases) + +```bash +$ curl -L https://github.com/golang-migrate/migrate/releases/download/$version/migrate.$platform-amd64.tar.gz | tar xvz +``` + +### MacOS + +```bash +$ brew install golang-migrate +``` + +### Linux (*.deb package) + +```bash +$ curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey | apt-key add - +$ echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/migrate.list +$ apt-get update +$ apt-get install -y migrate +``` + +### With Go toolchain + +#### Versioned + +```bash +$ go get -u -d github.com/golang-migrate/migrate/cmd/migrate +$ cd $GOPATH/src/github.com/golang-migrate/migrate/cmd/migrate +$ git checkout $TAG # e.g. v4.1.0 +$ go build -tags 'postgres' -ldflags="-X main.Version=$(git describe --tags)" -o $GOPATH/bin/migrate github.com/golang-migrate/migrate/cmd/migrate +``` + +#### Unversioned + +```bash +$ go get -tags 'postgres' -u github.com/golang-migrate/migrate/cmd/migrate +``` + +#### Notes + +1. Requires a version of Go that [supports modules](https://golang.org/cmd/go/#hdr-Preliminary_module_support). e.g. Go 1.11+ +1. These examples build the cli which will only work with postgres. In order +to build the cli for use with other databases, replace the `postgres` build tag +with the appropriate database tag(s) for the databases desired. The tags +correspond to the names of the sub-packages underneath the +[`database`](../database) package. +1. Similarly to the database build tags, if you need to support other sources, use the appropriate build tag(s). +1. Support for build constraints will be removed in the future: https://github.com/golang-migrate/migrate/issues/60 + +## Usage + +```bash +$ migrate -help +Usage: migrate OPTIONS COMMAND [arg...] + migrate [ -version | -help ] + +Options: + -source Location of the migrations (driver://url) + -path Shorthand for -source=file://path + -database Run migrations against this database (driver://url) + -prefetch N Number of migrations to load in advance before executing (default 10) + -lock-timeout N Allow N seconds to acquire database lock (default 15) + -verbose Print verbose logging + -version Print version + -help Print usage + +Commands: + create [-ext E] [-dir D] [-seq] [-digits N] [-format] NAME + Create a set of timestamped up/down migrations titled NAME, in directory D with extension E. + Use -seq option to generate sequential up/down migrations with N digits. + Use -format option to specify a Go time format string. + goto V Migrate to version V + up [N] Apply all or N up migrations + down [N] Apply all or N down migrations + drop Drop everyting inside database + force V Set version V but don't run migration (ignores dirty state) + version Print current migration version +``` + +So let's say you want to run the first two migrations + +```bash +$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2 +``` + +If your migrations are hosted on github + +```bash +$ migrate -source github://mattes:personal-access-token@mattes/migrate_test \ + -database postgres://localhost:5432/database down 2 +``` + +The CLI will gracefully stop at a safe point when SIGINT (ctrl+c) is received. +Send SIGKILL for immediate halt. + +## Reading CLI arguments from somewhere else + +### ENV variables + +```bash +$ migrate -database "$MY_MIGRATE_DATABASE" +``` + +### JSON files + +Check out https://stedolan.github.io/jq/ + +```bash +$ migrate -database "$(cat config.json | jq '.database')" +``` + +### YAML files + +```bash +$ migrate -database "$(cat config/database.yml | ruby -ryaml -e "print YAML.load(STDIN.read)['database']")" +$ migrate -database "$(cat config/database.yml | python -c 'import yaml,sys;print yaml.safe_load(sys.stdin)["database"]')" +``` diff --git a/cmd/migrate/examples/Dockerfile b/cmd/migrate/examples/Dockerfile new file mode 100644 index 0000000..c78b320 --- /dev/null +++ b/cmd/migrate/examples/Dockerfile @@ -0,0 +1,15 @@ +FROM ubuntu:bionic + +RUN apt-get update && \ + apt-get install -y \ + apt-transport-https \ + ca-certificates \ + curl \ + gnupg-agent + +RUN curl -sSL https://packagecloud.io/golang-migrate/migrate/gpgkey | apt-key add - +RUN echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ bionic main" > /etc/apt/sources.list.d/migrate.list +RUN apt-get update && \ + apt-get install -y migrate + +RUN migrate -version