Commit Graph

90 Commits

Author SHA1 Message Date
David Aurelio c8962a1d9b Fix packager breakages on node4
Summary:
This fixes issues that only surface when running with node4 / npm2.
Closes https://github.com/facebook/react-native/pull/11888

Differential Revision: D4415021

Pulled By: bestander

fbshipit-source-id: 8672cd892c9dae41cc7b5b7b3227eb24dd780d22
2017-01-13 09:28:41 -08:00
David Aurelio ae6cd177de create better debuggable source maps
Summary:
Introduces a new mechanism to build source maps that allows us to use real mapping segments instead of just mapping line-by-line.

This mechanism is only used when building development bundles to improve the debugging experience in Chrome.

The new mechanism takes advantage of a new feature in babel-generator that exposes raw mapping objects. These raw mapping objects are converted to arrays with 2, 4, or 5 for the most compact representation possible.
We no longer generate a source map for the bundle that maps each line to itself in conjunction with configuring babel generator to retain lines.

Instead, we create a source map with a large mappings object produced from the mappings of each individual file in conjunction with a “carry over” – the number of preceding lines in the bundle.

The implementation makes a couple of assumptions that hold true for babel transform results, e.g. mappings being in the order of the generated code, and that a block of mappings always belongs to the same source file. In addition, the implementation avoids allocation of objects and strings at all costs. All calculations are purely numeric, and base64 vlq produces numeric ascii character codes. These are written to a preallocated buffer objects, which is turned to a string only at the end of the building process. This implementation is ~5x faster than using the source-map library.

In addition to providing development source maps that work better, we can now also produce individual high-quality source maps for production builds and combine them to an “index source map”. This approach is unfeasable for development source maps, because index source map consistently crash Chrome.

Better production source maps are useful to get precise information about source location and symbol names when symbolicating stack traces from crashes in production.

Reviewed By: jeanlauliac

Differential Revision: D4382290

fbshipit-source-id: 365a176fa142729d0a4cef43edeb81084361e54d
2017-01-12 14:28:41 -08:00
Jean Lauliac 48a7ede4d2 packager: do not try to fetch uncached files
Reviewed By: davidaurelio

Differential Revision: D4403623

fbshipit-source-id: 8319e6bac365fe5fcebefd4f338faf145f662ecb
2017-01-12 10:43:32 -08:00
Jean Lauliac 7b5d796953 packager: remove validateOpts for Bundler class
Reviewed By: davidaurelio, cpojer

Differential Revision: D4380831

fbshipit-source-id: e3b5b2c39e83bf0b49028820e0f17daef27d7b3c
2017-01-06 06:13:54 -08:00
Jean Lauliac 8adb91f7e2 packager: centralize babelRegisterOnly callsites
Reviewed By: davidaurelio

Differential Revision: D4380987

fbshipit-source-id: db34e56b2025fb072798ef2cc3e52f3d1cb7c33c
2017-01-06 06:13:54 -08:00
David Aurelio ebf8731ae4 Revamp debug logging
Summary:
- Switch namespace prefix from `ReactNativePackager:` to `RNP:`
- Add logging for handled requests
- Log transform cache key
- Add logging for transform cache hits/misses

Reviewed By: jeanlauliac

Differential Revision: D4377867

fbshipit-source-id: 9ec2060432f8c5e68561d3fe8ec7127f76c4a081
2017-01-04 07:13:32 -08:00
Jean Lauliac 23261a6745 packager: Fix typing of Resolver options
Summary:
`declareOpts` is weakly typed. Since the callsite of Resolver constructor is itself flowifyed, we can get rid of `declareOpts`, and this provides us with much better typing.

Eventually I'd like to get rid of most of the defaults in the packager's inner code (ex. `watch` being false, `dev` being true). The reason is that defaults everywhere are prone to causing inconsistencies (for ex. some other code could have `dev` as false by default), problems that cannot be caught by Flow. Instead of having non-required options, I believe it is more sensible to provide helper functions that build sets of default options.

Reviewed By: davidaurelio, cpojer

Differential Revision: D4351874

fbshipit-source-id: 38653063f8939b4282c7c27cb6d5e3f3a25a9484
2016-12-21 03:43:36 -08:00
Christoph Pojer c383638e5e Remove `pit` and `mockImpl`
Reviewed By: dmitriiabramov

Differential Revision: D4321635

fbshipit-source-id: 460889a1f956f3733e7e49883dd97c9a8a561b86
2016-12-13 15:13:39 -08:00
Jean Lauliac d556e5824a packager: Resolver&co: @flow
Reviewed By: davidaurelio

Differential Revision: D4285448

fbshipit-source-id: ec9bc6fd3c6d6eae89347762bff8b5b24662394b
2016-12-08 05:13:40 -08:00
Jean Lauliac 6d26816cc6 packager: GlobalTransformCache: globalized keyOf
Reviewed By: davidaurelio

Differential Revision: D4243863

fbshipit-source-id: 917a8a116baf67c838c062b7b713dd4660d9f673
2016-11-30 07:43:42 -08:00
David Aurelio 7181a2d436 add flow
Summary: This adds flow types to new code written for the Buck/Packager integration.

Reviewed By: cpojer

Differential Revision: D4175156

fbshipit-source-id: 38c3d2c9176c7b3cf22b8baed7d445a75d033d04
2016-11-17 08:58:55 -08:00
David Aurelio 907f08a794 Use array lookup for localized dependency IDs
Summary:
The current transform for require calls replaces strings with module-local IDs. That means that each module would need a local require function.
To save hundreds of closure allocations, we can just use an array that maps local IDs to global IDs.

This diff changes the dependency collection and replacement transform to change a call like `require('React')` to something like `require(_dependencyMap[0])` rather than `require(0)`.

Reviewed By: cpojer

Differential Revision: D4153714

fbshipit-source-id: a63455834c6c2a75da6977cacb9aac9f2cb1b3aa
2016-11-17 08:58:55 -08:00
Jean Lauliac dfd9713fce packager worker: more @flow
Reviewed By: matryoshcow

Differential Revision: D4160528

fbshipit-source-id: 5f43cb47717288c5344c1204cf435ec727ca752e
2016-11-16 10:43:25 -08:00
Jean Lauliac 77d1ca470d packager: JSTransformer/index.js: @flow
Reviewed By: cpojer

Differential Revision: D4167006

fbshipit-source-id: db0deb2262fe2c53735cb46d9de6e6da1b375180
2016-11-15 04:28:53 -08:00
Jean Lauliac 2bf0c037ec packager worker: enable possibility to add Flow annotations
Reviewed By: matryoshcow

Differential Revision: D4160456

fbshipit-source-id: bbc2420126d8e8b725c0b7b359b35f09dfe8c8d6
2016-11-11 05:29:30 -08:00
Jean Lauliac ee637ee0eb packager worker/index.js: do not write to cache directly
Reviewed By: davidaurelio

Differential Revision: D4159970

fbshipit-source-id: 133ce5797e94a39fe60cbb9983edd561ce7d818e
2016-11-10 09:13:54 -08:00
Jean Lauliac 31c7108806 packager Module.js: fix concurrency issues
Reviewed By: matryoshcow

Differential Revision: D4159743

fbshipit-source-id: 41515f2a29344b648c4c52100056d2054e3edff7
2016-11-10 05:58:41 -08:00
Ovidiu Viorel Iepure b75dc61a46 Fix lint warnings in JSTransformer
Summary: Fix lint warnings throughout JSTransformer.

Reviewed By: bestander

Differential Revision: D4148161

fbshipit-source-id: ac73a9e3919b1c66fef0e447083b44039900f963
2016-11-09 09:31:12 -08:00
Ovidiu Viorel Iepure 5a6d3f99e2 Use native Promises
Summary: Use native Promises in React Native Packager. Remove all the non-standard `Promise.done()` calls throughout the codebase & replace `Promise.denodeify` with the stand-alone `denodeify` module.

Reviewed By: davidaurelio

Differential Revision: D4146965

fbshipit-source-id: 1730531c914863ac3c52626801d9f91c28eed717
2016-11-08 12:28:47 -08:00
Jean Lauliac c18d46bd03 JSTransformer/worker: cache transformed code as separate files
Reviewed By: davidaurelio, cpojer

Differential Revision: D4051477

fbshipit-source-id: 0179ce18dd20c00083ae05b5cf5f925659e7c056
2016-11-08 03:43:37 -08:00
Ovidiu Viorel Iepure 1ac86b2304 File transform events
Summary: Correctly generate log entries for file transforms and ping telemetry using the new Logger API.

Reviewed By: bestander, cpojer

Differential Revision: D4081325

fbshipit-source-id: 7d50d54eb673a0276512db6ad5ff21c344495612
2016-11-03 07:44:10 -07:00
Ovidiu Viorel Iepure 4d23c1f423 Instrumentation API
Reviewed By: cpojer

Differential Revision: D4074413

fbshipit-source-id: fd2dff17168f426680fd9dc8456f8b1ae3f55318
2016-11-03 07:44:10 -07:00
Jean Lauliac d60ac14fd0 extract-dependencies.js: fix out-of-date comment
Summary: We actually use the AST, not regexes. Fix comment.

Reviewed By: davidaurelio

Differential Revision: D4104797

fbshipit-source-id: e37d168bd541d9d222667d0d168aa6b6099c1275
2016-10-31 11:43:37 -07:00
Ovidiu Viorel Iepure dc0f7875c8 Add details to Activity events
Summary: Update Activity API to allow adding more details to events for telemetry purposes.

Reviewed By: davidaurelio

Differential Revision: D3982691

fbshipit-source-id: 07f3ed5d1ec4eddbbdeb00feb02ea75e1168705e
2016-10-12 10:43:59 -07:00
David Aurelio 47a7a9ee80 Add `js_file_prod` rule
Reviewed By: matryoshcow

Differential Revision: D3913439

fbshipit-source-id: fe3a30fddb4d8fcabfc88caf9b1b032189812b89
2016-10-05 09:28:50 -07:00
David Aurelio 85b74ab639 Add `js_file` rule
Reviewed By: matryoshcow

Differential Revision: D3900455

fbshipit-source-id: 61384ade035db978ef3ca258e4c0109bcb450692
2016-10-05 09:28:50 -07:00
Ovidiu Viorel Iepure 351d9da0e8 Add file transform event to Activity
Summary: Add a telemetric event to the transformer.

Reviewed By: davidaurelio

Differential Revision: D3950902

fbshipit-source-id: c46e95ee93925fb0f9d1784aebdc3206baf74dc6
2016-09-30 08:43:34 -07:00
Cristian Carlesso bb80ef6326 Removing automatically reset modules between test run
Reviewed By: bestander

Differential Revision: D3886125

fbshipit-source-id: c8f47c3466add1e2c89649a1c6f47b01f0d7a89e
2016-09-19 12:43:55 -07:00
David Aurelio 3263f18a6e Adapt jest transform for node-only files
Summary:
This changes the jest preprocessor so that files targetet at node.js will be run with the node-specific transform.
It also adapts tests that relied on inline requires.

Benefit: packager tests run faster now.

Reviewed By: cpojer

Differential Revision: D3562007

fbshipit-source-id: e06c86d545926a5c546458025f505dca115e7ea8
2016-07-15 06:28:26 -07:00
Johannes Stein 81961d8756 Fixes typo in error message
Summary:
This pull request fixes a small typo when a module fails being transformed through the React Packager.
Closes https://github.com/facebook/react-native/pull/8596

Differential Revision: D3522272

Pulled By: mkonicek

fbshipit-source-id: e117a26bab5a99573ac68fb0e7618df0a14325a4
2016-07-06 08:13:45 -07:00
David Aurelio dee7aadd28 inline `Platform.select`
Summary: We are already inlining `Platform.OS`. This diff adds support to inline calls to `Platform.select` with an object literal as first argument. The transform will replace the call with the property value corresponding to the platform, or `undefined` if it does not exist.

Reviewed By: frantic

Differential Revision: D3385391

fbshipit-source-id: bb068d17948ed84e381707faeaa0450399c2f306
2016-06-05 05:58:20 -07:00
David Aurelio 5f53658fb0 remove support for `process.platform`
Reviewed By: javache

Differential Revision: D3252666

fb-gh-sync-id: af0e4987c34f43ec2472cbdf52fc112b81050513
fbshipit-source-id: af0e4987c34f43ec2472cbdf52fc112b81050513
2016-05-04 02:51:26 -07:00
David Aurelio 17726e1ae6 Improve constant inlining, add `process.platform`
Reviewed By: bestander

Differential Revision: D3235716

fb-gh-sync-id: f9019ec0042827e409fa84ba74f4c426ccad1519
fbshipit-source-id: f9019ec0042827e409fa84ba74f4c426ccad1519
2016-04-28 11:23:21 -07:00
Christoph Pojer d4e704be17 Update Jest APIs on fbsource
Reviewed By: javache

Differential Revision: D3229435

fb-gh-sync-id: b0e252d69e1f399a946fca6e98ef62ff44c2ef9c
fbshipit-source-id: b0e252d69e1f399a946fca6e98ef62ff44c2ef9c
2016-04-27 19:16:32 -07:00
Mike Grabowski ae9f74e68d Strip shebang when present in JS files
Summary:Fixes #7034
Closes https://github.com/facebook/react-native/pull/7073

Differential Revision: D3199816

fb-gh-sync-id: 2099dd1f81b030933794be6a592a697cec3627d0
fbshipit-source-id: 2099dd1f81b030933794be6a592a697cec3627d0
2016-04-19 18:09:22 -07:00
Kureev Alexey 4a5cbbbec8 Replace underscore by lodash
Summary:As far as we agreed to merge `rnpm` into react-native core, we need to align our dependencies to exclude duplications. One of the steps forward would be to use the same utilities library. According to the thread on fb, everybody is fine with replacing underscore by lodash (which we use internally for rnpm).

So, here we go!

cc mkonicek davidaurelio grabbou

**Test plan**
```
$ npm test
```
![image](https://cloud.githubusercontent.com/assets/2273613/13173972/ee34c922-d700-11e5-971b-68ff7322b6d6.png)

**Code formatting**

Changes are aligned with the current [style guide](https://github.com/facebook/react-native/blob/master/CONTRIBUTING.md#style-guide).
Closes https://github.com/facebook/react-native/pull/6030

Differential Revision: D3016271

Pulled By: davidaurelio

fb-gh-sync-id: c4f6776a7de7470283d3ca5a8b56e423247f5e45
shipit-source-id: c4f6776a7de7470283d3ca5a8b56e423247f5e45
2016-03-09 03:09:44 -08:00
David Aurelio faaafd771e Make uglify not append //# sourceMappingURL=
Summary: We also need a more recent version of uglify that supports this

Reviewed By: martinbigio

Differential Revision: D3024959

fb-gh-sync-id: f9efdddceda4f726567c39884c844a8e74e6e09d
shipit-source-id: f9efdddceda4f726567c39884c844a8e74e6e09d
2016-03-08 12:22:31 -08:00
David Aurelio 3e1708bcc3 transform before extracting dependencies
Summary:Make packager transform files before extracting their dependencies.

This allows us to extract dependencies added by transforms (and to avoid including them manually).

It also allows for better optimization and to get rid of the “whole program optimization” step:
This diff utilizes the new worker introduced in D2976677 / d94a567 – that means that minified builds inline the following variables:

- `__DEV__` → `false`
- `process.env.NODE_ENV` → `'production'`
- `Platform.OS` / `React.Platform.OS` → `'android'` / `'ios'`

and eliminates branches of conditionals with constant conditions. Dependency extraction happens only after that step, which means that production bundles don’t include any modules that are not used.

Fixes #4185

Reviewed By: martinbigio

Differential Revision: D2977169

fb-gh-sync-id: e6ce8dd29d1b49aec49b309201141f5b2709da1d
shipit-source-id: e6ce8dd29d1b49aec49b309201141f5b2709da1d
2016-03-08 09:51:26 -08:00
David Aurelio d13b75794f limit number of workers
Summary:This limits the number of spawned transformation workers depending on the number of available cores (`n`):

```
n < 3   → n
n < 8   → floor(3/4 * n)
n < 24  → floor(3/8 * n + 3) // factor going from 3/4 to 1/2
n >= 24 → floor(n / 2)
```

Reviewed By: bestander

Differential Revision: D2999894

fb-gh-sync-id: 0163240b5f066432f9ba07161c0dfa2ec767ba58
shipit-source-id: 0163240b5f066432f9ba07161c0dfa2ec767ba58
2016-03-02 07:31:42 -08:00
David Aurelio 5309991ba6 Add new worker for code transform, optimization, and dependency extraction
Summary:This adds a new worker implementation that

- uses the existing transforms to transform code
- optionally inline `__DEV__`, `process.env.NODE_ENV`, and `Platform.OS`
- optionally eliminate branches of conditionals with constant conditions
- extracts dependencies
- optionally minifies

This will land as part of a multi-commit stack, not in isolation

Reviewed By: martinbigio

Differential Revision: D2976677

fb-gh-sync-id: 38e317f90b6948b28ef2e3fe8b66fc0b9c75aa38
shipit-source-id: 38e317f90b6948b28ef2e3fe8b66fc0b9c75aa38
2016-03-01 04:41:32 -08:00
David Aurelio aafa9d684d Upgrade node-haste to v2.3.0
Summary: This updates to the latest published node-haste version, and also adapts code and tests to that version. Future upgrades should be easier.

Reviewed By: bestander

Differential Revision: D2963144

fb-gh-sync-id: 9fd2c84fc49643fb85ee5d9674a5e458d43d44ca
shipit-source-id: 9fd2c84fc49643fb85ee5d9674a5e458d43d44ca
2016-02-23 06:10:33 -08:00
Christoph Pojer 60dacf3bc4 Install node-haste2 and remove DependencyResolver
Summary: This installs the 2.0 version of node-haste, removes the DependencyResolver and fixes up all the tests.

Reviewed By: davidaurelio

Differential Revision: D2943416

fb-gh-sync-id: aa83d436a33f910d12ed4cc6e2ad8d5742c123a5
shipit-source-id: aa83d436a33f910d12ed4cc6e2ad8d5742c123a5
2016-02-18 18:03:34 -08:00
Adam Miskiewicz 38cea2edeb Use "babel-preset-react-native"
Summary:
Rather than specifying Babel plugins in the `.babelrc` packaged with react-native, leverage a Babel preset to define the plugins (https://github.com/exponentjs/babel-preset-react-native).

This allows for a much better user experience for those who want (or need) to override options in their project's `.babelrc`.

Prior to this PR, if a user wanted to use a custom babel-plugin (or a custom set of babel plugins), they'd have either 1) manually override the `.babelrc` in the react-packager directory (or fork RN), or 2) specify a custom transformer to use when running the packager that loaded their own `.babelrc`. Note - the custom transformer was necessary because without it, RN's `.babelrc` options would supersede the options defined in the project's `.babelrc`...potentially causing issues with plugin ordering.

This PR makes the transformer check for the existence of a project-level `.babelrc`, and if it it's there, it _doesn't_ use the react-native `.babelrc`. This prevents any oddities with Babel plug
Closes https://github.com/facebook/react-native/pull/5214

Reviewed By: davidaurelio

Differential Revision: D2881814

Pulled By: martinbigio

fb-gh-sync-id: 4168144b7a365fae62bbeed094d8a03a48b4798c
2016-02-03 08:15:32 -08:00
Martín Bigio e8472b44f4 Move HMR to external transformer
Summary:
public

`babel-plugin-react-transform` doesn't support to run on transformed code (it doesn't detect some react components). The reason why HMR was originally on the internal transform was so that it worked with non JS code (TypeScript, Coffeescript, etc), but looks like this is not very popupar in the community, maybe because the JS ecosystem is getting better everyday.

So long story short, for now lets move HMR to the external transformer. This should also give us a perf boost.

Reviewed By: davidaurelio

Differential Revision: D2870973

fb-gh-sync-id: 68ca8d0540b88b9516b9fef10643f93fc9b530d2
2016-01-28 17:02:32 -08:00
Martín Bigio da3838f6ca Unit test `worker.js`
Summary:
public

We had a bug a few weeks ago which caused transform errors not to be shown properly. The problem was on this piece of code, so lets unit test it.

Reviewed By: davidaurelio

Differential Revision: D2849990

fb-gh-sync-id: 9f6bb8a8b7d59bbaa3577c29cee37fb23a50d66f
2016-01-21 09:10:37 -08:00
Martín Bigio 62a3d2c843 Make Hot Loading work on OSS
Summary:
public

- Tweak OSS server to enable the HMR connection
- Remove client gating code.
- Resolve internal transforms plugins

After this diff, Hot Loading should work on OSS.

Reviewed By: javache

Differential Revision: D2803620

fb-gh-sync-id: b678180c884d2bfaf454edf9e7abe6b3b3b32ebe
2016-01-14 19:33:42 -08:00
Martin Bigio a401b1d166 Bump timeout to track intermittent error
Reviewed By: sam-swarr

Differential Revision: D2816605

fb-gh-sync-id: 3d05746493f0ff97c396c82eb30d9a49537a0473
2016-01-08 13:39:28 -08:00
Martín Bigio 0cc1132b8c Add option to disable internal transforms
Reviewed By: davidaurelio

Differential Revision: D2815307

fb-gh-sync-id: 17ed8f13de7b4c41111efa4a5f2af5283e6ef3e0
2016-01-08 11:10:29 -08:00
David Aurelio 87e8086937 Require transform file eagerly in transform worker
Summary:
One consequence we didn't predict after introducing the Internal Transform Pipeline, was that when the workers would get started, we won't require the external transformer the user specified up until the first time each worker received a job. There're 2 visible consequences of this: (1) the transform progress bar seems to get stuck for about 5 seconds the first time the packager receives a request and (2) the first N (# of cores) HMR requests take way longer (about 4 seconds with FB's transformer instead of << 1 second) as we need to require lots of modules.

This diff creates a temporary file for the js transformer workers that requires the user-specified transform file eagerly.
That makes sure workers have imported babel and the transforms before receiving the first request.

There are better ways to do this, like adding an `init()` method to the workers and call that eagerly. I will follow with another diff doing that.

public

Reviewed By: javache

Differential Revision: D2812153

fb-gh-sync-id: 15be316b792d1acd878ed9303bea398aa0b52e1d
2016-01-08 08:35:32 -08:00
Martín Bigio db91ff81fd Bail internal transforms if there're no transforms to run
Reviewed By: sam-swarr

Differential Revision: D2810474

fb-gh-sync-id: d5ba48922c91c3dbacfccb0d087cc1deac14fa1c
2016-01-06 20:04:27 -08:00