Commit Graph

143 Commits

Author SHA1 Message Date
Amjad Masad 4a423d7bba [react-packager] Fix error in template string and bump timeout
Summary:
Fix error in the template string (no plus, thinks it's a function).
And bump the timeout to 30 seconds because a file is taking more than 10 seconds `js/RKJSModules/Libraries/FBComponents/FBFed/FeedStoryFragments.js`
2015-08-20 15:29:38 -07:00
Martín Bigio 2330fa6bce [react-packager] Integration test for `runServerHere.sh` 2015-08-20 12:00:06 -07:00
Martín Bigio 06eb63b54d [react-packager] Introduce bundle IDs and keep track of parent/child
Summary:
Since JS doesn't have the guarantee that once a bundle is loaded it will stay in memory (and this is something we actually don't want to enforce to keep memmory usage low), we need to keep track of parent/child relationships on the packager to pass it down to native. As part of this diff, we also introduced an ID for each bundle. The ID for a child bundle is shynthetized as the bundleID of the parent module + an index which gets incremented every time a new bundle is created. For instance given this tree:

       a,b
    c       f
  d   e       g

the ID for `d` will be `bundle.0.1.2`, the one for e will be `bundle.0.1.3` and the one for `g` will be `bundle.0.5.6`. This information will be useful to figure out which bundles need to be loaded when a `require.ensure` is re-written.
2015-08-20 10:19:09 -07:00
Amjad Masad 54a8fe9156 [react-packger] Add a timeout on transform jobs
Summary:
There's been a case where Babel can hang indefinitely on a file parse/transform. Possibly related to https://github.com/babel/babel/issues/2211

This adds a timeout to transform jobs and throws an error informing the user of the offending file. The timeout interval defaults to 10 seconds, but can be changed via an option.
2015-08-19 17:42:28 -07:00
Amjad Masad c5b2dcaf9d [react-packager] Remove unnecessary 'chalk' module mock 2015-08-19 17:28:18 -07:00
Amjad Masad 208f54002f [react-packager] Make the fs mock async 2015-08-19 17:17:15 -07:00
Amjad Masad 866df6d37c [react-packager] Wait for haste map before accepting any requests
Summary:
D2319999 introduced a regression where we stopped waiting for the "build haste map" step to finish before we accept any requests. This makes sure that we block on that.

Need to unbreak with this, but will follow up with a test to catch this in the future.
2015-08-19 16:17:37 -07:00
Martín Bigio d8c8993758 [react-packager] Fix bug on Bundles Layout algorithm
Summary:
The layout algorithm wasn't getting deep into the sync dependencies recursively to async dependencies.
2015-08-19 14:20:38 -07:00
Alex Kotliarskyi 90ca49cbd6 [RN] Unbreak packager 2015-08-18 18:29:32 -07:00
Martín Bigio 02b8f264ef [react-packager] Modernize Activity to ES6 2015-08-18 15:25:18 -07:00
Martín Bigio f77f48c61c [react-packager] Modernize AssetServer to ES6 2015-08-18 14:38:40 -07:00
Ben Alpert 801dd2d133 [ReactNative] Don't redbox for React warnings when not using Chrome executor 2015-08-17 16:02:10 -07:00
Amjad Masad 3555afb66d [reat-packager] Switch platform resolution based on query param
Summary:
Currently the platform selection is controlled by the blacklist. However, since we want to use the same server instance for cross-platform development, we need this to be controlled per request.

One outstanding issue, is that the DependencyGraph class wasn't designed that way and it doesn't have a per-request state. This means that with the current design race conditions is possible. If we got a request for a different platfrom while processing the previous request, we may change the outcome of the previous request.

To fix this a larger refactor is needed. I'll follow up a diff to do that.

Finally, so I don't break the universe like last time, I'll leave it up to the RN guys to update the call sites.
2015-08-15 13:28:18 -07:00
Alex Kotliarskyi 44b97249e9 [ReactNative] Add ability to listen for Packager events 2015-08-14 14:28:45 -08:00
Martín Bigio daf56c32eb [react-packager] Modernize `Server` to ES6 2015-08-14 12:34:23 -08:00
Martín Bigio fec5d8d4d4 [react-packager] Cache in which bundle is each module
Summary:
Not that at the moment a module can be present in multiple bundles, so the new API will return only one of them. In the near future we'll impose the invariant that a module can only be present in a single bundle so this API will return the exact bundle in which it is.
2015-08-14 12:30:30 -08:00
Martín Bigio c3be08e66b [react-packager] Modernize `Server-test` by using ES6 features 2015-08-14 12:23:02 -08:00
Amjad Masad 7664d5fb76 [react-packager] Use module objects across the codebase (rid of getPlainObject etc)
Summary:
Instead of using plain objects and having to convert to and from them we just use the `Module` class across the codebase.
This seems cleaner and can enforce the type as opposed to fuzzy objects.
2015-08-13 15:52:31 -08:00
Amjad Masad a372af9956 [react-packager] Remove horribly outdated example_project
Summary:
Removes some old unused code.
2015-08-13 13:02:52 -08:00
Martín Bigio 8644bfd09c [react-packager] Introduce Bundler
Summary:
Introduce a Bundler capable of generating the layout of modules for a given entry point. The current algorithm is the most trivial we could come up with: (1)it puts all the sync dependencies into the same bundle and (2) each group of async  dependencies with all their dependencies into a separate bundle. For async dependencies we do this recursivelly, meaning that async dependencies could have async dependencies which will end up on separate bundles as well.

The output of of the layout is an array of bundles. Each bundle is just an array for now with the dependencies in the order the requires where processed. Using this information we should be able to generate the actual bundles by using the `/path/to/entry/point.bundle` endpoint. We might change the structure of this json in the future, for instance to account for parent/child bundles relationships.

The next step will be to improve this algorithm to avoid repeating quite a bit dependencies across bundles.
2015-08-13 12:57:55 -08:00
Amjad Masad 1751d1c6fc [react-packager] Set a lifetime on workers to avoid memory leaks 2015-08-12 19:26:26 -08:00
Martín Bigio ce6acf1586 [react-packager] Introduce `require.ensure`
Summary:
This is the first step to add support for splitting the JS bundle into multiple ones. This diff adds support for keeping track of the async dependencies each module has. To do so we introduce the following syntax:

  require.ensure(['dep1', 'dep2, ..., 'depN'], callback);

Where the callback function is asynchronously invoked once all the indicated modules are loaded.

Internally, the packager keeps track of every set of async dependencies a module has. So for instance if a module looks like this:
  require.ensure(['dep1'], () => {...});
  require.ensure(['dep2'], () => {...});

the `Module` object will keep track of each set of dependencies separately (because we might want to put them on separate bundles).
2015-08-12 12:23:42 -08:00
Amjad Masad 3f1619b158 [react-packager] Rename 'Package' to 'Bundle'
Summary:
The word Package is overloaded, it may mean npm package, or may mean a collection of bundles. Neither is what we mean. We mean `bundle`.

This renames it and modernize some of the Bundler code.
2015-08-12 12:09:01 -08:00
Martín Bigio d1772eebe6 [react-packager] Fix Cache-test 2015-08-11 11:27:15 -08:00
Evan Solomon d1dc802e20 [Babel] Upgrade babel and regenerator to the latest version
Summary:
See c0fd4c1f9e
Closes https://github.com/facebook/react-native/pull/1753
Github Author: Evan Solomon <evan@evanalyze.com>

@allow-crlf-text
2015-08-10 20:37:39 -08:00
Martín Bigio 8c521f9749 [react-packager] Add more granular Activity logging 2015-08-10 16:25:04 -08:00
Martín Bigio 49fde903b8 [react-packager] Promote Cache to top level
Summary:
The cache is only used for JSTransformer at the moment. We're doing IO and some computation to get each module's name, whether is a haste or node module and it's dependencies. This work happens on startup so by caching this value we shouldbe able to reduce the start up time. Lets promote the Cache to the Packager level to be able to use it by any of the components of the packager. For now, on this diff we'll start using it to cache the mentioned fields.

Also we had to introduce the concept of fields in the cache as manually merging the date we had for each path is not possible as we're using promisses all around. With the new API, each field is a promise.

@amasad and I did some manual testing to measure the impact of this change and looks like it's saves 1 second when building the haste map (which represents 50% of the time). Overall this reduces 1 second of start up time which was currently about 8s on my mac book pro.
2015-08-10 16:25:03 -08:00
Martín Bigio 06f508a55a [JSAppServer] Don't keep track of not found packages 2015-08-06 11:34:56 -08:00
Thomas Aylott f47a29f922 [react-packager] Fixes stack traces
Summary:
TransformErrors weren't showing stack traces, making it hard to debug problems in transformer code.
2015-08-05 12:46:30 -08:00
James Ide 7b1ffed4e7 [Async] Enable async/await and update UIExplorer and tests
Summary:
- Enables async/await in .babelrc and transformer.js
- Adds regenerator to package.json. Users still need to explicitly require the regenerator runtime -- this is so that you only pay for what you use.
- Update AsyncStorage examples in UIExplorer to use async/await
- Update promise tests in UIExplorer to use async/await in addition to the promise API

Closes https://github.com/facebook/react-native/pull/1765
Github Author: James Ide <ide@jameside.com>
2015-08-04 05:35:13 -08:00
Amjad Masad 63a96af6c6 [react-packager] Add support for platform in the resolver
Summary:
Teach the resolver about platform-based resolution. The platform extension is inferred from the entry point.
It works for haste modules, as well as node-based resolution.
2015-08-03 18:29:20 -08:00
James Ide 03b4050c6e [Packager] Include Content-Type headers with bundle and source maps
Summary:
The packager did not send back the Content-Type headers. Adding these.

Closes https://github.com/facebook/react-native/pull/2029
Github Author: James Ide <ide@jameside.com>
2015-07-24 18:46:17 -08:00
Dave Sibiski f7d51c750f [Packager] Fix when loading a path that can't be handled
Summary:
[Packager] Adds `NotFoundError` when loading a path that can't be handled

Resolves https://github.com/facebook/react-native/issues/1944
Closes https://github.com/facebook/react-native/pull/1948
Github Author: Dave Sibiski <dsibiski@gmail.com>
2015-07-15 08:25:31 -08:00
Amjad Masad 802ad3feb3 [react-packager] fix test 2015-07-01 17:37:51 -08:00
James Ide 62431c7f62 [Tests] Update tests to run on io.js with the latest version of jest
Summary:
[This is a preview diff for getting RN's tests to pass with a future version of jest that supports io.js and other future versions of Node. This can be merged once the diff to update jest is merged upstream and published.]

Updates the tests in small ways so they run on io.js with two updates:

 - The Cache test which relies on Promises uses `runAllImmediates` for modern versions of Node because bluebird uses `setImmediate` instead of `process.nextTick` for Node >0.10.

Closes https://github.com/facebook/react-native/pull/1382
Github Author: James Ide <ide@jameside.com>

Test Plan:  Run `npm test` with the latest version of jest.
2015-07-01 13:02:29 -08:00
Amjad Masad 70dde863ae [react-packager] Use latest babel-core in place of babel (40% perf improvement) 2015-06-30 03:55:21 -08:00
Amjad Masad 6b2c12e300 [react-packager] Update sane to get a new version of fb-watchman (perf) 2015-06-26 17:48:39 -08:00
Amjad Masad 15a0f07c5f [react-packager] Enable watchman fs crawl
Summary:
@public
Now that watchman perf issue was fixed we can enable watchman-based fs crawling which is faster than node.
This showed an existing issue with some files missing from the blacklist which I addressed.

Test Plan:
./fbrnios.sh run
click around and scroll all the apps
2015-06-26 16:20:50 -08:00
Amjad Masad 737a32417f [react-packager] Injectible file crawlers (2x crawl speedup) 2015-06-24 18:04:57 -08:00
Jing Chen 693fd0bbe5 [events] Add JS require time and fix up some existing logs 2015-06-24 16:59:00 -08:00
Amjad Masad 95800d1564 [react-packager] Fix the confused node_modules detection function
Summary:
@public

We have a function that detects whether a give file is to be treated as a node_modules. If so it doesn't have access to the haste module map. There is an exception to this rule which is a few modules that are allowed to do that. Currently thats react-native, react-tools, and parse.

The current implementation had a bug where if you had `react-native` (or react-tools etc) in the name before the actual package root then the detection will be off. This fixes the problem by starting from the `lastIndexOf('node_modules')` directory, that way nothing confuses us.

Test Plan:
./runJestTests.sh
export OSS, patch, run e2e test
2015-06-24 15:20:35 -08:00
Christopher Chedeau fef3c7294d [ReactNative] Bring in node console.log formatting
Summary:
@public

The current output of console.log is extremely bad. If you pass NaN, it shows up as null (super confusing I know -_-), if you pass a cyclical object, it just says that it is cyclic and that's it. It doesn't print up the first few levels which are NOT cyclical and would be really helpful.

It turns out that nodejs console.log pretty printer is really awesome and can be easily extracted as a few hundred lines. This is going to be such a productivity boost that I think it's the right tradeoff to embed it like this

Test Plan:
```
var a = {kikoo: {lol: 1}}
a.kikoo.nice = a;

console.log(a);
> { kikoo: { lol: 1, nice: [Circular] } }

console.log(NaN)
> NaN
```
2015-06-24 14:11:55 -08:00
Forbes Lindesay bc1f94b7b2 Replace bluebird with promise 2015-06-22 08:44:03 -08:00
Bill Fisher 050f0b3558 [ReactNative][easy] fix server 500 response typo
Summary:
@public
corrected small typo in the 500 response from the packager server

Test Plan: add throw to promise function prior to error handler, run packager, cache a bundle with bundle extension URI, open /debug/packages, see clean 500 error
2015-06-20 16:43:18 -08:00
Amjad Masad 695a5c2fbc [react-packager] Cache in-memory file lookups (~200ms win on file change) 2015-06-19 22:47:26 -08:00
Amjad Masad 8faa406e96 [react-packager] Rewrite dependency graph (support node_modules, speed, fix bugs etc)
Summary:
@public
Fixes #773, #1055
The resolver was getting a bit unwieldy because a lot has changed since the initial writing (porting node-haste).
This also splits up a large complex file into the following:

* Makes use of classes: Module, AssetModule, Package, and AssetModule_DEPRECATED (`image!` modules)
* DependencyGraph is lazy for everything that isn't haste modules and packages (need to read ahead of time)
* Lazy makes it fast, easier to reason about, and easier to add new loaders
* Has a centralized filesystem wrapper: fast-fs (ffs)
* ffs is async and lazy for any read operation and sync for directory/file lookup which makes it fast
* we can easily drop in different adapters for ffs to be able to build up the tree: watchman, git ls-files, etc
* use es6 for classes and easier to read promise-based code

Follow up diffs will include:
* Using new types (Module, AssetModule etc) in the rest of the codebase (currently we convert to plain object which is a bit of a hack)
* using watchman to build up the fs
* some caching at the object creation level (we are recreating Modules and Packages many times, we can cache them)
* A plugin system for loaders (e.g. @tadeuzagallo wants to add a native module loader)

Test Plan:
* ./runJestTests.sh react-packager
* ./runJestTests.sh PackagerIntegration
* Export open source and run the e2e test
* reset cache
* ./fbrnios.sh run and click around
2015-06-19 18:05:18 -08:00
Amjad Masad 0405c82798 [react-packager] Cache based on options, not url
Summary:
@public

We cached based on url, which wasn't unique becuase some options would be defaulted. This was obvious when starting the server via fbrnios which tries to warmup the bundle.
And then when the device woke up it will send a request (that is identical in reality) but would miss the cache.

This changes the cache key into a JSON stringification of the options.

Test Plan:
* ./runJestTests.sh
* ./fbrnios.sh run
2015-06-19 15:14:31 -08:00
Amjad Masad 4b9d5f8ad0 [react-packager] Add tests to ensure we return all dependency types 2015-06-18 21:27:55 -08:00
Amjad Masad 253544cb45 [react-packager] Make it safe to include files without a newline at the end
Summary:
@public
Fixes #1431
Fixes #1005
Files with no newlines and a comment at the end of the file would've caused a syntax error in the bundle:

```js
__d('module', function() {
hi();
// wow })
```

This fixes the issue by inserting a new lines before `})`.

Test Plan:
* ./runJestTests.sh
* ./runJestTests.sh PackagerIntegration
* open app to the playground app
* add an error
* observe that the redbox has the correct lines
2015-06-16 12:02:49 -08:00
Tadeu Zagallo 24026b6c25 [ReactNative] Revert packager ignoring node_modules 2015-06-11 10:45:32 -08:00