status-im
/
migrate
mirror of https://github.com/status-im/migrate.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
802 Commits 5 Branches 25 Tags 1.2 MiB
Commit Graph

181 Commits

Author SHA1 Message Date
Dale Hui 52fc08fdf5 Update supported DB versions 2019-01-12 16:13:26 -08:00
Dale Hui 2feaaaddda Update dktest from v0.2.0 to v0.3.0 
- Update ReadyFuncs accordingly
 2019-01-08 22:39:26 -08:00
Dale Hui bbec63fda7 Use one of the 2 ports exposed by MySQL Docker images 
- MySQL driver tests should pass more consistently now
 2019-01-08 05:11:07 -08:00
Dale Hui 809c7f0ac8 Use dktest to run docker tests 
- Leaving migrate/testing in case there are unknown consumers
    - Add migrate/dktesting package
    - Update tests to use migrate/dktesting instead of migrate/testing
 2019-01-08 05:11:07 -08:00
Dale Hui fc8d57d128
Merge branch 'master' into mongodb-driver 2019-01-07 22:50:51 -08:00
DBobrov 12b93a3a07 Change checking of mongo replica set initialization 2019-01-07 15:17:35 +03:00
DBobrov 3fa0df7553 Refactor mongodb transactions test 2019-01-07 11:47:37 +03:00
DBobrov 13978a108a Add test for migrations in transaction mode 2019-01-06 12:45:22 +03:00
DBobrov 30d2d94a22 Fix review comments 2019-01-05 11:10:25 +03:00
DBobrov 436b81b357 Replaced bsonx.Doc with bson.D 2019-01-04 22:01:21 +03:00
DBobrov 309edb6da6 - Add dependency for dep 
- Add transaction mode for migrations
 2019-01-03 18:00:23 +03:00
DBobrov bcd4f6e7dd added mongodb driver 2019-01-01 11:14:27 +03:00
DBobrov 26064905d4 Changed migrations table struct of field `version` to fix support `database.NilVersion` and default version format. 2018-12-28 15:53:55 +03:00
Dale Hui 7f00868584
Merge pull request #128 from andrei-m/fix-redshift 
Fix Redshift migrations driver
 2018-11-14 10:32:07 -08:00
Dale Hui 6ebc6c7dae Unlock in postgres db driver should also use the schema name 2018-11-14 02:18:57 -08:00
Dale Hui 06d3ebde1c Use strings.NewReader() instead of bytes.NewReader() when source is a string 
- Group imports
 2018-11-05 22:50:08 -08:00
Evgeniy Bastrykov d2d449ad78 Review comments 2018-11-05 16:03:54 +04:00
Evgeniy Bastrykov 16d63e3a76 Add support for multi-schema migrations in Postgres 
There is lock conflict on parallel migrations in different postgres
schemas. To avoid this conflicts function GenerateAdvisoryLockId added
variadic params to change lock id with schema name. Schema name taked
with postgres CURRENT_SCHEMA function. Null byte used as separator
between database and schema name, because any other symbol may be used
in both of it.

Closes #118
 2018-11-05 12:56:38 +04:00
Andrei Mackenzie cc573a74dc Remove unused errors 2018-11-03 19:27:55 -04:00
Andrei Mackenzie 1d54cf5f18 Replace 'redshift' with 'redshift2' 
This addresses https://github.com/golang-migrate/migrate/issues/90 . The
exported Redshift object no longer exports an embedde 'Driver' however,
so some more work is needed to make this backwards compatible.
 2018-11-03 19:01:37 -04:00
Andrei Mackenzie 4e098f74cd Use 'DELETE FROM' over 'TRUNCATE' in Redshift 
TRUNCATE commits the current transaction, which breaks the expection of
the 'Commit()' that follows.

See:
https://github.com/golang-migrate/migrate/issues/90
https://docs.aws.amazon.com/redshift/latest/dg/r_TRUNCATE.html
 2018-11-03 18:52:09 -04:00
Andrei Mackenzie b0300df5e7 Use the most Redshift-like Postgres version available on Dockerhub 
Redshift is based on Postgres version 8.0.2.
 2018-11-03 18:49:31 -04:00
Andrei Mackenzie 9c132fe622 Update README with Redshift-specific info 2018-11-03 18:47:12 -04:00
Andrei Mackenzie 22334834ac Support the 'redshift' URL scheme 
This brings the 'redshift2' package in alignment with the existing
'redshift' package.
 2018-11-03 18:44:11 -04:00
Andrei Mackenzie 3373343f23 Remove advisory locking from the Redshift implementation 
Redshift does not support advisory lock functions. The closest
capability is in-transaction table locks, which aren't quite right here
because the transaction scope is established within SetVersion, not
higher up above the Lock-before/Unlock-after SetVersion.

Local locking is left intact to satisfy expected "can't Lock twice
before Unlocking" behavior asserted in shared tests.
 2018-11-03 18:31:41 -04:00
Andrei Mackenzie 1d0bedc454 Rename 'Postgres' to 'Redshift' in 'redshift2' 2018-11-03 18:09:53 -04:00
Andrei Mackenzie 8381ea0307 Clone the 'postgres' driver as 'redshift2' 
Avoid stepping on the 'redshift' driver for the time being. Driver
registration is also modified to identify the clone as 'redshift2'
rather than 'postgres'.
 2018-11-03 18:04:22 -04:00
Ryo Takashima ca888b96e3 Add NewDB to spanner.go 2018-10-28 22:00:05 +09:00
Wessel Oosthuizen bc28cee1dc Add support for TLS on Cassandra. 2018-10-16 11:18:38 +02:00
Dale Hui fc2b2e9c64 Use v4 for Go module support 2018-10-10 16:16:23 -07:00
Dale Hui 107db3f407 Revert "Remove support for Go modules" 
This reverts commit 5e96539f55.
 2018-10-10 16:15:33 -07:00
Dale Hui 5e96539f55 Remove support for Go modules 2018-10-10 13:34:03 -07:00
Mike Fitzgerald 0d4f1c0212 Fix postgres drop 2018-10-04 01:22:39 -04:00
Dale Hui 16dea48c82 Fix module support by opting in to modules 2018-09-14 01:14:42 -07:00
Kay-Zee 1e4c50b70d
Add sql.DB to Postgres and Mysql structs to allow closing in Close() 2018-09-06 17:16:02 -07:00
Tatsuhito KATO 02c83fb38b more descriptive. 2018-08-15 02:42:18 +00:00
Tatsuhito KATO 9049e49f9e `x-tls-cert` and `x-tls-key` must set both. 2018-08-14 07:43:14 +00:00
Tatsuhito KATO 9d36dd9841 Add detail of `tls` query. 2018-08-14 07:42:10 +00:00
Tatsuhito KATO 9905791932 Fixes MySQL custom TLS connection failure. 2018-08-13 11:43:19 +00:00
Dale Hui dd00ca926e Add support for postgres 10 and drop support for postgres 9.2 
https://www.postgresql.org/support/versioning/
 2018-08-12 15:54:21 -07:00
Anton Markelov df7ce674e9 add support for the multiline clickhouse queries 2018-08-03 16:08:47 +10:00
Dale Hui df658e8fa8 Don't urlencode the username and password for MySQL 
Addresses: https://github.com/golang-migrate/migrate/pull/69
 2018-07-24 17:56:17 -07:00
Dale Hui b5edb8e50c Improve error messaging when URL parsing fails 
- Improve docs around escaping/encoding reserved URL characters
    - Add tests for net/url Parse() and reserved characters to document understanding
    - Addresses: https://github.com/golang-migrate/migrate/issues/77
 2018-07-24 17:55:58 -07:00
Dale Hui 97eb0d191b Fix broken GenerateAdvisoryLockId() test 
- Add it to the test suite to ensure that it doesn't break again
 2018-07-24 17:55:58 -07:00
Dale Hui 857d7a620d Revert "Merge pull request #69 from bcho/fix/mysql-url-password-encode" 
This reverts commit 78d696c1e5, reversing
changes made to 18583d5a91.
 2018-07-24 17:55:58 -07:00
Alan Graham 4244ebea6a Add detail to README for Cassandra flag, improve if block 2018-07-24 19:30:44 +00:00
Alan Graham 327822c29a Added Cassandra URL flag to enable multi statement migrations 2018-07-20 13:27:54 +00:00
Alan Graham c4b81d1210 Support for multiple statements in Cassandra cql migration files based on semi-colon and newline. 2018-07-17 15:19:54 +00:00
Dale Hui b742ec3de4 Use unix line endings 2018-07-09 19:21:40 -07:00
Damir Vandic ae2fd4fa7b Support postgres non-ascii error parsing 2018-07-09 11:39:29 +02:00
Damir Vandic 3eb26a65d3 Add postgres lib/pq error parsing 2018-07-07 12:17:38 +02:00
Chao Deng 48a4062f75 prevent net/url encoding the user password 2018-06-29 21:07:10 +08:00
Dale Hui c4e855b58c List source and database drivers in help text 2018-06-12 16:17:52 +08:00
Teddy Schmitz 046d1cb767 postgres driver should wait for lock 2018-06-01 09:11:24 +08:00
Dale Hui a890d44d2f CockroachDB driver improvements 
- Correctly return database.ErrLocked when the DB is locked
    - Group imports
    - gofmt goodness
 2018-05-31 14:44:48 -07:00
kenjones 1512e41e41 Minor tweaks to remove duplication 
Adds missing connection close for cassandra tests

Revert to default timeout of 600ms
 2018-05-16 20:09:12 -04:00
Dale Hui 55a25c5e0e Cassandra DB driver test improvements 
- shorten connect timeout to allow more connection retry attempts
    - close session after it's created
    - group imports
 2018-05-16 15:12:36 -07:00
Andrés Rodríguez d607064fd2 Switch Cassandra's package lock to instance lock 2018-05-16 16:26:23 -03:00
Andrés Rodríguez 8b0f876416 Rename Cassandra's WithSession to WithInstance 2018-05-16 16:23:21 -03:00
Andrés Rodríguez bbd8f410d9 Correct method name in comment 2018-05-16 10:54:56 -03:00
Andrés Rodríguez 78c47074a3 Add WithSession helper for Cassandra driver 
Also changes method receivers names for the Cassandra struct to "c"
since "p" makes no sense in this context.
 2018-03-13 15:49:08 -03:00
Dale Hui 22f249514d Merge branch 'postgres-lock-fix' 
Addresses: https://github.com/golang-migrate/migrate/pull/4
 2018-02-20 16:12:54 -08:00
Dale Hui 1519c59188 Rename Postgres.db to Postgres.conn 2018-02-20 15:40:20 -08:00
Dale Hui ab7dfb38ac Merge remote-tracking branch 'origin/master' into postgres-lock-fix 2018-02-20 15:32:34 -08:00
Reed Allman 094bad431e s/db/conn/ 2018-02-09 15:14:05 -08:00
Reed Allman a8ef2b8cfe update CI and build tags to go1.9 2018-02-09 00:32:47 -08:00
Reed Allman 636c911d33 fixes mysql lock failure 
I believe this closes #297 as well.

I have been working on adding testing of migrations and it requires acquiring
the lock in mysql multiple times to go up and down. After nailing this down to
GET_LOCK returning a failure for every subsequent GET_LOCK call after the
first, I decided it was time to rtfm and lo and behold:

https://dev.mysql.com/doc/refman/5.7/en/miscellaneous-functions.html#function_release-lock

RELEASE_LOCK will not work if called from a different thread than GET_LOCK.
The simplest solution using the golang database/sql pkg appears to be to just
get a single conn to use for every operation. since migrations are happening
sequentially, I don't think this will be a performance hit (possible
improvement). now calling Lock() and Unlock() multiple times will work;
prior to this patch, every call to RELEASE_LOCK would fail. this required
minimal changes to use the *sql.Conn methods instead of the *sql.DB methods.

other changes:

* upped time out to 10s on GET_LOCK, 1s timeout can too easily leave us in a
state where we think we have the lock but it has timed out (during the
operation).
* fixes SetVersion which was not using the tx it was intending to, and fixed a
bug where the transaction could have been left open since Rollback or Commit
may never have been called.

I added a test but it does not seem to come up in the previous patch, at least
easily, I tried some shenanigans. At least, this behavior should be fixed with
this patch and hopefully the test / comment is advisory enough.

thank you for maintaining this library, it has been relatively easy to
integrate with and most stuff works (pg works great :)
 2018-02-09 00:27:01 -08:00
Dale Hui 24dd870559 MySQL and Postgres db driver test improvements 
- Log unexpected connection errors while waiting for the docker image to start
    - Don't leave connections open
 2018-01-20 00:11:35 -08:00
Dale Hui 51ec421ad8 Remove debugging print from mysql driver test 2018-01-19 15:13:07 -08:00
Dale Hui 1cf095c51d Update more references to github.com/mattes/migrate 2018-01-19 14:59:27 -08:00
Dale Hui 7c9a261dc0 Fix MySQL docker image connection testing 2018-01-19 14:50:31 -08:00
Dale Hui f584949470 Update imports to reference fork. 
Thanks `make rewrite-import-paths`!
 2018-01-19 10:56:55 -08:00
Jason Hutchinson c156f5f3a1 replace TRUNCATE with DELETE FROM to avoid schema changes within transaction 2017-11-02 16:59:26 -05:00
Teddy Schmitz 155a8b241c Use db.Conn to fix postgres lock/unlock 2017-10-27 10:56:10 +08:00
Ashod Ayanyan 28cb030ffe Add support for multiple DDL/SQL statements in one migration file 2017-09-18 15:43:46 -07:00
Jens Rantil a5e584ce22 feat(cassandra): support for user/pw authentication 2017-09-17 13:32:00 +02:00
Jens Rantil d25732a53a sty: avoid unnecessary parenthesis 
Golang best-practise. Cleanup made by `gofmt`.
 2017-09-17 13:30:08 +02:00
Jens Rantil 701bd68731 style(cassandra): whitespace fixes 
According to `gofmt`.
 2017-09-17 13:26:30 +02:00
Jens Rantil 51acd3f3ad style: order imports lexicographically 
Best-practise according to `gofmt`.
 2017-09-17 13:26:23 +02:00
Taylor Wrobel 2c0f4f588e Update cockroachDB Txn signatures 
cockroach-go made a backwards-incompatible change to their transaction
function signatures, which was causing the cockroach instrumentation
to fail.

Updates the signature to match the cockraoch-go change.
 2017-08-10 22:56:41 -07:00
Taylor Wrobel 40a40613cb Add cockroachdb README and add ForceLock functionality 
Adds a readme to the cockroachdb database package, following the
postgres readme style.

Also adds the ability to force acquisition of the migration lock
via a connect URL parameter/WithInstance config, to allow for fixing
cases where an implementation error causes the schema lock to not
be released.

Lastly, tweaks the CLI readme to include information on building a
CLI for databases other than postgres.
 2017-07-11 21:52:02 -07:00
Watanabe Masaru 23a2745a3a fix spanner ensure migration table 2017-07-09 10:57:10 -07:00
Kirill Shvakov 1f62e3f142 Merge branch 'master' into master 2017-07-08 13:50:51 +03:00
Taylor Wrobel 2cab359222 Remove debugging output from cockroachDB tests 2017-07-07 23:13:55 -07:00
Taylor Wrobel c2925c40c7 Add tests for CockroachDB 2017-07-07 23:12:07 -07:00
Taylor Wrobel 88115dedbf Support running docker with commands and handling multiple exposed ports 
Adds the ability to specify a series of commands to run as part
of the docker image execution, and allows for retrieving a mapping
of an exposed via the port bound within the container.
 2017-07-07 23:12:07 -07:00
Taylor Wrobel 1d8a881941 Add CockroachDB Support 
Adds support for CockroachDB.  Cockroach uses the postges wire
protocol and has a large amount of common SQL functionality shared
with Postgres, so much of the postgres code was able to be copied
and modified.

Since the protocol is used in determining the driver, and the
Postgres protocol is also used by Cockroach, new connect string
prefixes were added: cockroach:// cockroachdb:// and
crdb-postgres://.  These fake protocol strings are replaced in
the connect function with the correct `postgres://` protocol.

TODO: Tests needed (Cockroach has a docker image, so this shouldn't
be too hard)
 2017-07-07 23:12:07 -07:00
kshvakov 94056e51cc postgres: fix SetVersion. All requests were executed in different connections so commit/rollback doesn't work correctly 2017-06-23 11:13:54 +03:00
Kirill Shvakov 8cd9761d29 fix: set current database 2017-06-21 23:10:43 +03:00
kshvakov e86f4a201c change include path from github.com/kshvakov to github.com/mattes 2017-06-21 17:24:55 +03:00
kshvakov 9341404de0 move ensureVersionTable to init 2017-06-21 17:14:43 +03:00
kshvakov b031b22c80 Add WithInstance 2017-06-21 17:12:05 +03:00
kshvakov 1440a070c3 Add WithInstance 2017-06-21 17:06:04 +03:00
kshvakov 14636b77fb Add ClickHouse driver 2017-06-21 15:58:46 +03:00
Christian Klotz eccc3a26d2 Skip test if SPANNER_DATABASE isn’t set 2017-06-20 17:53:41 +01:00
Ben 13a818e3b5 Fix typo forgotton => forgotten 2017-06-14 21:59:11 +02:00
Fuyuan Bie 39bb76224d Fix parenthesis problem. 2017-06-14 21:57:37 +02:00
Fuyuan Bie 9632882b69 Use both db name and migration table name for lock hash 2017-06-14 21:57:37 +02:00
Christian Klotz 803598293e Add basic Google Spanner test 2017-06-14 19:46:10 +02:00
Jolan Malassigne 89879968bb add cassandra driver and function to retrieve networkSettings to get port bound to 9042 2017-06-05 11:03:49 +01:00