Summary:
The goal of this pull request is to make it easier for contributors to run Android tests locally, specifically the unit tests and integration tests. I added a bunch of checks to the local testing scripts that will warn you if your environment is misconfigured, and tell you how to fix it. I also updated the testing docs, so that the regular "Testing" page should be a decent resource to point people to when you are telling them "hey this pull request needs a test." Just Android, though, I haven't gotten to the iOS parts yet.
I also disabled a couple tests that seemed quite flaky while running on a local machine, and don't seem to be providing much value. In particular, the `TestId` test just hangs on my emulator a lot and has been flaky on CI in the past, so I removed about half of its test cases to make the sample app smaller. The testMetions test appears to be dependent on screen size so I commented it out.
Closes https://github.com/facebook/react-native/pull/11442
Differential Revision: D4323569
Pulled By: bestander
fbshipit-source-id: 9c869f3915d5c7cee438615f37986b07ab251f8c
Summary:
This is the manually imported version of https://github.com/facebook/react-native/pull/10786
This was mostly straigthforward by replacing the local-cli folder with the version I had in my local git checkout,
plus a few other files I listed with git diff --name-only.
Reviewed By: hramos
Differential Revision: D4201118
fbshipit-source-id: 4d0fb54b0edda9de1abba427958e420fd2ac105c
Summary:
This should stop Circle CI from crashing.
Let's wait for the test to pass.
cc davidaurelio matryoshcow mkonicek
Closes https://github.com/facebook/react-native/pull/10962
Differential Revision: D4186949
Pulled By: kentaromiura
fbshipit-source-id: 2542376a6e4bf0d7dc423c574bd4573ae9bdfc96
Summary: This is a simple change to allow not specifying xcodebuild's `-destination` flag if we just need to build the project without running the test.
Reviewed By: bestander
Differential Revision: D4173546
fbshipit-source-id: 0b77bb5fc01d8cb6a8c9bc765294796631839655
Summary:
Force npm for e2e tests.
Bonus: tracked down and removed a non-standard `.done()` call that was breaking e2e tests.
Reviewed By: bestander
Differential Revision: D4153514
fbshipit-source-id: 079939874ea75c7ea6e3e5376d37873c95fc74c1
Summary:
- switched to yarn on travis (experimental)
- removed xcpretty, we need more verbose logs
- returned jest test for e2e
Closes https://github.com/facebook/react-native/pull/10442
Differential Revision: D4043991
Pulled By: davidaurelio
fbshipit-source-id: 50062f6a77a17643bf9a7ba7add8cee5b3bb26d2
Summary:
- Consolidate common code in iOS and tvOS test scripts
- Start the packager before starting tests, to improve reliability
- Increase timeout value in RCTTestRunner.m
Closes https://github.com/facebook/react-native/pull/10378
Differential Revision: D4028364
Pulled By: bestander
fbshipit-source-id: 24c2124a1c62643a02f0668b60a67b971e08d1a3
Summary:
* Motivation
Fix a possible issue with Travis CI by running tvOS tests in a separate script.
Closes https://github.com/facebook/react-native/pull/10268
Differential Revision: D3980775
Pulled By: matryoshcow
fbshipit-source-id: 66dc52f4c0e5c492eb51d53fba209fc25361ce3e
Summary:
* Motivation *
Second PR for Apple TV support.
* Test plan *
Apple TV tests have been added to scripts/objc-test.sh
Closes https://github.com/facebook/react-native/pull/10227
Differential Revision: D3974064
Pulled By: javache
fbshipit-source-id: 36dffb4517efa489e40fa713a30655d1d76ef646
Summary:
Currently too much noise is coming because of broken tests
Closes https://github.com/facebook/react-native/pull/10083
Differential Revision: D3919029
Pulled By: davidaurelio
fbshipit-source-id: e75bb38d4be1c8427dfa0c3f6df60d04582ddb2c
Summary: Automatically tags and publish them upstream, less steps required in order to do the release rather than just `bump-oss-version.js`
Differential Revision: D3913826
Pulled By: bestander
fbshipit-source-id: 6e023e7ab7b486ff6e6cc8e946f19e10a7ceeef8
Summary:
Adding jest and its presets to the react-native init command
**Test plan (required)**
run react-native init foo (using `npm link` to use the local `react-native` version)
inside foo there are now a .babelrc file and the package.json is set up as described by
https://facebook.github.io/jest/docs/tutorial-react-native.html#setup
Closes https://github.com/facebook/react-native/pull/9719
Differential Revision: D3843037
Pulled By: bestander
fbshipit-source-id: 004e27ebd3f257a202ed43f378d6fe6cc23ced52
Summary:
- kill -9 SERVER_PID does not work for packager currently because it is started as daemon.
- And lego tests just hang until they are killed e.g. intern/sandcastle/1952254070/187417721/
- fixed bezier test because it annoyed me with random breaks because of precision
Reviewed By: davidaurelio
Differential Revision: D3528588
fbshipit-source-id: 87e5b4330fa69bc9a8a7f48e2250f3c2239f2b35
Summary:
Android folder may contain versions from previous runs and it may affect e2e test execution.
This should make tests independent of cached jars.
Closes https://github.com/facebook/react-native/pull/8277
Differential Revision: D3462901
fbshipit-source-id: 25b1f75b9b207617431e26ad60f5d4de322ab1f4
Summary:
Some changes are required to this scripts in order to successfully run it on Sandcastle:
- the packager needs to be started before the test, doing this only when not running in Travis
- no xcpretty available, piping to anything else (e.g. `cat`) fails
Closes https://github.com/facebook/react-native/pull/8127
Differential Revision: D3437268
Pulled By: avaly
fbshipit-source-id: a0a52f08b31f50b88f5d0fd4d9d6c827df179d71
Summary:
Some changes are required to this scripts in order to successfully run it on Sandcastle:
- the packager needs to be started before the test
- no xcpretty available, piping to anything else (e.g. `cat`) fails
Closes https://github.com/facebook/react-native/pull/8123
Differential Revision: D3436833
Pulled By: avaly
fbshipit-source-id: e284d6eaf370720b9a0f2468b114a596028f7107
Summary:
Some changes are required to this scripts in order to successfully run it on Sandcastle:
- the packager needs to be started before the test
- no xcpretty available, piping to anything else (e.g. `cat`) fails
Closes https://github.com/facebook/react-native/pull/8123
Differential Revision: D3436833
Pulled By: avaly
fbshipit-source-id: 893ac5cba4197cf5bb6b492701171910fd10f99d
Summary:
Add more flexibility to the iOS UIExplorer test scripts:
- support environments without `xcpretty` installed
- support custom xcode destination argument
Initial PR is based on the last git commit which passed on Travis today (2a92b52) in order to test Travis build.
I'll rebase to latest, in order to fix merge conflicts, before merging.
**Test plan (required)**
Make sure tests pass on Travis.
Closes https://github.com/facebook/react-native/pull/7982
Differential Revision: D3404574
Pulled By: avaly
fbshipit-source-id: 48aabd81fba67d482af46728a9c3975842f03060
Summary:
- TestIdTestModule instrumentation tests is responsible for too many occasional crashes, e.g. https://circleci.com/gh/facebook/react-native/7054.
This should fix the problem (will monitor over next week)
- Made file naming more consistent
- 5 retries don't make e2e tests more stable, reduced back to 3 but I need to investigate how to make it more reliable
Closes https://github.com/facebook/react-native/pull/7784
Differential Revision: D3354444
fbshipit-source-id: d058362edbec09522a4828998e01988a82a74487
Summary:
Fixes npm deployment script as in https://circleci.com/gh/facebook/react-native/6745.
Example situation:
- admin is ready to release 0.26.0
- admin adds tag v0.26.0 and pushes to origin
- Circle CI build fails because of code error
- admin cherry-picks a fix and tags v0.26.0 again and pushes to origin
- Circle does not update local tag v0.26.0 because it already has it cached, compiles the code but fails to deploy to npm because v0.26.0 is not on branch HEAD from Circle point of view
The previous version of the script was checking the tags on current commit as well but it used the local branch tags.
This change fetches tags from `origin` fresh and allows us to redeploy the same version multiple times.
Closes https://github.com/facebook/react-native/pull/7619
Differential Revision: D3334469
fbshipit-source-id: b423fc19516dc4f5f7f2605224e62b8a378c8a7d
Summary:
Fixed regex that looks for OK sign
**Test plan (required)**
- circle CI results
Closes https://github.com/facebook/react-native/pull/7410
Differential Revision: D3265016
fb-gh-sync-id: 1eba7297116d31f8bf3393733141058cf6a7454a
fbshipit-source-id: 1eba7297116d31f8bf3393733141058cf6a7454a
Summary:
This change makes all instrumentation tests to be executed in sequence in independent retriable processes.
With a new test being open sourced recently our CI stability degraded.
This PR should bring back stability because tests won't affect each other and will have shorter lifetime
Closes https://github.com/facebook/react-native/pull/7353
Differential Revision: D3259081
fb-gh-sync-id: 48ccdb5dbd561d416526497ff474378db9ca3c60
fbshipit-source-id: 48ccdb5dbd561d416526497ff474378db9ca3c60
Summary:Implemented smarter retries for e2e tests
E2e tests consist of two expensive and flaky steps: npm installation and tests execution.
Our CI is configured to retry a command 3 times before terminating the test.
In current setup if one of the steps fail the whole test is restarted.
This change adds ad-hoc ability to retry flaky bits of e2e script independently.
This will make tests fail faster when code gets broken while increasing the chances to succeed in case of random false errors.
Closes https://github.com/facebook/react-native/pull/7184
Differential Revision: D3218927
fb-gh-sync-id: 9be8343484bb28aa3601b651db70fc55aa840e61
fbshipit-source-id: 9be8343484bb28aa3601b651db70fc55aa840e61
Summary:The Obj-C tests were stalling when we updated TravisCI to use the OS X image with Xcode 7.3 and iOS 9.3.
On my own MBP with Xcode 7.3 the tests stall as well. Running `./scripts/objc-test.sh` prints out a warning near the end, which says `ld` couldn't find a Frameworks folder under the iPhoneSimulator9.3 SDK directory. Indeed, this directory doesn't exist on my computer. When creating a brand new project with unit tests and UI tests, both test targets don't specify "Framework Search Paths" so I don't think we need to specify it anymore.
Closes https://github.com/facebook/react-native/pull/7168
Differential Revision: D3216524
fb-gh-sync-id: 3097bda98d78289ecdba86801bcd461f3311ac47
fbshipit-source-id: 3097bda98d78289ecdba86801bcd461f3311ac47
Summary:Quite a few of these steps are now tested automatically on Android thanks to bestander.
**Test plan**
Just visual inspection of the code :)
Closes https://github.com/facebook/react-native/pull/7037
Differential Revision: D3191678
fb-gh-sync-id: 54702a772c5c149bdd3f05d6f4ca257bc6f27090
fbshipit-source-id: 54702a772c5c149bdd3f05d6f4ca257bc6f27090
Summary:Packager usually builds within 250 seconds and sometimes misses test timeout.
Also added non-persistent option to make it build faster.
Should be fine because we don't test live reloading in ios
Closes https://github.com/facebook/react-native/pull/6974
Differential Revision: D3178884
fb-gh-sync-id: e9bf75456f1cf480ffea6aa2c769550b40eb3d32
fbshipit-source-id: e9bf75456f1cf480ffea6aa2c769550b40eb3d32
Summary:- converted shell script `scripts/e2e-test.sh` into JS script to have more programming flexibility
- using appium execute 2 tests after a fresh React Native app installation: check HMR and that debugging mode does not crash the app
- made sure tests can be stable on limited CI systems and added ways to debug any problems in the future
Using appium we can now interact with Android app and test its state.
As a follow up i am planning to write a blog post on how to use appium with android and ios for e2e testing.
Closes https://github.com/facebook/react-native/pull/6840
Differential Revision: D3173635
Pulled By: mkonicek
fb-gh-sync-id: 3cf044bc9f64d1a842ae4589dd1bcab76de3d66a
fbshipit-source-id: 3cf044bc9f64d1a842ae4589dd1bcab76de3d66a
Summary:BUCK is faster than Gradle.
For example `gradle app:installDebug` vs `buck install app` is ~7 seconds vs ~2 seconds with warm caches.
This is just the beginning to allow people to become familiar with BUCK.
It is enough for running the app locally and testing on a device.
Gradle is still used for dependency resolution.
Closes https://github.com/facebook/react-native/pull/6733
Differential Revision: D3126328
Pulled By: bestander
fb-gh-sync-id: 56aad276036c029af7e0e23d60c46ba2f77b3d2c
fbshipit-source-id: 56aad276036c029af7e0e23d60c46ba2f77b3d2c
Summary:npm-publish.js is not cohesive enough: besides building and publishing it also modifies some files.
It is better to have a separate script that will bump versions, make a commit and tag it.
scripts/bump-oss-version.js does exactly that.
This simplifies release process and allows manual release to npm if CI is not available.
Closes https://github.com/facebook/react-native/pull/6625
Differential Revision: D3092849
fb-gh-sync-id: 92cf38bd3df31c8c9c433fc5f9e15c129855fe0e
shipit-source-id: 92cf38bd3df31c8c9c433fc5f9e15c129855fe0e
Summary:This work allows automated release deployment.
Previous semi-automation lived in release.sh and I split it into two pieces:
- test-manual-e2e.sh - that just tests that current commit is buildable and makes a quick e2e installation for manual testing
- publish-npm.js - that makes publish based on what current branch and tags are on commit that is tested/deployed by CI
This simplified `Releases.md` guide and requires you to just run
```
git checkout -b 0.22-stable
git tag v0.22.0-rc
git push origin 0.22-stable --tags
```
to have a successful npm release.
Closes https://github.com/facebook/react-native/pull/6453
Reviewed By: mkonicek
Differential Revision: D3047938
Pulled By: bestander
fb-gh-sync-id: dbebf4c3a0bc2c2a0ef75c54595ab5654f91b8ea
shipit-source-id: dbebf4c3a0bc2c2a0ef75c54595ab5654f91b8ea
Summary:Running CI tests with emulator, not a graddle wrapper.
This should make things faster
Closes https://github.com/facebook/react-native/pull/6421
Differential Revision: D3042505
fb-gh-sync-id: 455264f90044c22ed3ec8cf9cb6d5edc2efedcdb
shipit-source-id: 455264f90044c22ed3ec8cf9cb6d5edc2efedcdb
Summary:Need to make sure we can use both versions of npm client for publishing and installing
**Test plan (required)**
Make sure tests pass on both Travis and Circle CI.
Closes https://github.com/facebook/react-native/pull/6364
Differential Revision: D3036537
fb-gh-sync-id: a3c1ea16f0aeb7b1327574b511aac7594fd3d7e6
shipit-source-id: a3c1ea16f0aeb7b1327574b511aac7594fd3d7e6
Summary:The e2e-test.sh currently only supports iOS. Most of the script will be identical for Android,
let's therefore reuse it. This diff adds one argument: --ios, --android or --packager.
`--packager` starts the packager and checks it produces a bundle. The idea is to run this
on CircleCI even before we implement the actual Android e2e test in order to reliably catch
breakages to package.json quickly.
We should only land this once Travis is green again.
**Test plan**
Ran:
git checkout master
./scripts/e2e-test.sh # Error message about required argument was printed
./scripts/e2e-test.sh --ios # Ran the test as before (all the way up to xctool which I don't have locally)
./scripts/e2e-test.sh --android # Message was printed
./scripts/e2e-test.sh --packager # The packager created the bundle, exit code was 0
# Made the packager fails on bad JS
./scripts/e2e-test.sh --packager # Exit code was 1
Closes https://github.com/facebook/react-native/pull/6279
Differential Revision: D3007281
fb-gh-sync-id: 6ece06b933001ba0939806c69ed7c7471b134931
shipit-source-id: 6ece06b933001ba0939806c69ed7c7471b134931
Summary:quick fix to get CI stable
Closes https://github.com/facebook/react-native/pull/6271
Differential Revision: D3005703
fb-gh-sync-id: 88941f2bb058d65637a06a3710108cdf131c3bda
shipit-source-id: 88941f2bb058d65637a06a3710108cdf131c3bda
Summary:Added ability to run instrumentation tests with BUCK.
This change uses BUCK to build and run instrumentation tests facebook style.
The gains are that we can execute the same tests internally at FB and in OSS.
Also running tests not via graddle:connect command is 1.5 minutes faster.
I'll keep keep an eye on stability Gradle and BUCK builds for a while.
Closes https://github.com/facebook/react-native/pull/6176
Differential Revision: D2999878
Pulled By: bestander
fb-gh-sync-id: d715ba231769e57100685a1256f2e530c589921c
shipit-source-id: d715ba231769e57100685a1256f2e530c589921c
Summary:
Out of 57 builds in the 5 days during Christmas only 15 were successful.
15 of the failed ones were caused by `com.android.ddmlib.ShellCommandUnresponsiveException` when running unstrumentation tests.
Another 10 were because of crashes probably related to memory usage.
This PR follows ideas from https://github.com/Originate/guide/blob/master/android/guide/Continuous%20Integration.md
So far I've made 6 successful builds with this setup.
Need to run it for a few more days to get accurate stats.
Closes https://github.com/facebook/react-native/pull/5021
Reviewed By: svcscm
Differential Revision: D2795713
Pulled By: androidtrunkagent
fb-gh-sync-id: 33373fed7ca7c5fb83b35cf551f8501286e33d7b
Internally we don't mirror `scripts` folder from GitHub, which means
when importing #3523 changes to `scripts/e2e-test.sh` will not make it to
GitHub. This commit contains those changes.
This way it's possible to re-run e2e tests faster without re-downloading
all packages. `npm unpublish` is used to avoid getting warning about
publishing the same version of the package
Kevin Lacker
Emil Sjolander
Tim Yung
Martin Konicek
Konstantin Raev
Kevin Gozali
Ovidiu Viorel Iepure
Cristian Carlesso
dlowder-salesforce
Mike Grabowski
kentaromiura
Pieter De Baets
Valentin Agachi
Valentin Agachi
James Ide
Ben Alpert
Konstantin Raev
Martin Konicek
Pieter De Baets
Satyajit Sahoo
Martin Bigio
Alex Kotliarskyi
Felix Oghină
Felix Oghină
Tadeu Zagallo
Christopher Chedeau
