Commit Graph

126 Commits

Author SHA1 Message Date
Amjad Masad bc28a35bda [react-packager] Use module objects across the codebase (rid of getPlainObject etc)
Summary:
Instead of using plain objects and having to convert to and from them we just use the `Module` class across the codebase.
This seems cleaner and can enforce the type as opposed to fuzzy objects.
2015-08-13 15:52:31 -08:00
Amjad Masad abce124198 [react-packager] Remove horribly outdated example_project
Summary:
Removes some old unused code.
2015-08-13 13:02:52 -08:00
Martín Bigio 5cad2e9370 [react-packager] Introduce Bundler
Summary:
Introduce a Bundler capable of generating the layout of modules for a given entry point. The current algorithm is the most trivial we could come up with: (1)it puts all the sync dependencies into the same bundle and (2) each group of async  dependencies with all their dependencies into a separate bundle. For async dependencies we do this recursivelly, meaning that async dependencies could have async dependencies which will end up on separate bundles as well.

The output of of the layout is an array of bundles. Each bundle is just an array for now with the dependencies in the order the requires where processed. Using this information we should be able to generate the actual bundles by using the `/path/to/entry/point.bundle` endpoint. We might change the structure of this json in the future, for instance to account for parent/child bundles relationships.

The next step will be to improve this algorithm to avoid repeating quite a bit dependencies across bundles.
2015-08-13 12:57:55 -08:00
Amjad Masad 7750db05e6 [react-packager] Set a lifetime on workers to avoid memory leaks 2015-08-12 19:26:26 -08:00
Martín Bigio cfcf604b3d [react-packager] Introduce `require.ensure`
Summary:
This is the first step to add support for splitting the JS bundle into multiple ones. This diff adds support for keeping track of the async dependencies each module has. To do so we introduce the following syntax:

  require.ensure(['dep1', 'dep2, ..., 'depN'], callback);

Where the callback function is asynchronously invoked once all the indicated modules are loaded.

Internally, the packager keeps track of every set of async dependencies a module has. So for instance if a module looks like this:
  require.ensure(['dep1'], () => {...});
  require.ensure(['dep2'], () => {...});

the `Module` object will keep track of each set of dependencies separately (because we might want to put them on separate bundles).
2015-08-12 12:23:42 -08:00
Amjad Masad 309326db2e [react-packager] Rename 'Package' to 'Bundle'
Summary:
The word Package is overloaded, it may mean npm package, or may mean a collection of bundles. Neither is what we mean. We mean `bundle`.

This renames it and modernize some of the Bundler code.
2015-08-12 12:09:01 -08:00
Martín Bigio 70624dc347 [react-packager] Fix Cache-test 2015-08-11 11:27:15 -08:00
Evan Solomon 51ccdfa36a [Babel] Upgrade babel and regenerator to the latest version
Summary:
See c0fd4c1f9e
Closes https://github.com/facebook/react-native/pull/1753
Github Author: Evan Solomon <evan@evanalyze.com>

@allow-crlf-text
2015-08-10 20:37:39 -08:00
Martín Bigio a303a42e28 [react-packager] Add more granular Activity logging 2015-08-10 16:25:04 -08:00
Martín Bigio 2afeb8fbe6 [react-packager] Promote Cache to top level
Summary:
The cache is only used for JSTransformer at the moment. We're doing IO and some computation to get each module's name, whether is a haste or node module and it's dependencies. This work happens on startup so by caching this value we shouldbe able to reduce the start up time. Lets promote the Cache to the Packager level to be able to use it by any of the components of the packager. For now, on this diff we'll start using it to cache the mentioned fields.

Also we had to introduce the concept of fields in the cache as manually merging the date we had for each path is not possible as we're using promisses all around. With the new API, each field is a promise.

@amasad and I did some manual testing to measure the impact of this change and looks like it's saves 1 second when building the haste map (which represents 50% of the time). Overall this reduces 1 second of start up time which was currently about 8s on my mac book pro.
2015-08-10 16:25:03 -08:00
Martín Bigio aefdf82cdc [JSAppServer] Don't keep track of not found packages 2015-08-06 11:34:56 -08:00
Thomas Aylott a6ff11f1d2 [react-packager] Fixes stack traces
Summary:
TransformErrors weren't showing stack traces, making it hard to debug problems in transformer code.
2015-08-05 12:46:30 -08:00
James Ide 47e1d1aef8 [Async] Enable async/await and update UIExplorer and tests
Summary:
- Enables async/await in .babelrc and transformer.js
- Adds regenerator to package.json. Users still need to explicitly require the regenerator runtime -- this is so that you only pay for what you use.
- Update AsyncStorage examples in UIExplorer to use async/await
- Update promise tests in UIExplorer to use async/await in addition to the promise API

Closes https://github.com/facebook/react-native/pull/1765
Github Author: James Ide <ide@jameside.com>
2015-08-04 05:35:13 -08:00
Amjad Masad 18452940f0 [react-packager] Add support for platform in the resolver
Summary:
Teach the resolver about platform-based resolution. The platform extension is inferred from the entry point.
It works for haste modules, as well as node-based resolution.
2015-08-03 18:29:20 -08:00
James Ide 53222f0dda [Packager] Include Content-Type headers with bundle and source maps
Summary:
The packager did not send back the Content-Type headers. Adding these.

Closes https://github.com/facebook/react-native/pull/2029
Github Author: James Ide <ide@jameside.com>
2015-07-24 18:46:17 -08:00
Dave Sibiski 70feab9d50 [Packager] Fix when loading a path that can't be handled
Summary:
[Packager] Adds `NotFoundError` when loading a path that can't be handled

Resolves https://github.com/facebook/react-native/issues/1944
Closes https://github.com/facebook/react-native/pull/1948
Github Author: Dave Sibiski <dsibiski@gmail.com>
2015-07-15 08:25:31 -08:00
Amjad Masad b45e2ed7ed [react-packager] fix test 2015-07-01 17:37:51 -08:00
James Ide 5aa27586f0 [Tests] Update tests to run on io.js with the latest version of jest
Summary:
[This is a preview diff for getting RN's tests to pass with a future version of jest that supports io.js and other future versions of Node. This can be merged once the diff to update jest is merged upstream and published.]

Updates the tests in small ways so they run on io.js with two updates:

 - The Cache test which relies on Promises uses `runAllImmediates` for modern versions of Node because bluebird uses `setImmediate` instead of `process.nextTick` for Node >0.10.

Closes https://github.com/facebook/react-native/pull/1382
Github Author: James Ide <ide@jameside.com>

Test Plan:  Run `npm test` with the latest version of jest.
2015-07-01 13:02:29 -08:00
Amjad Masad 7d184adf1a [react-packager] Use latest babel-core in place of babel (40% perf improvement) 2015-06-30 03:55:21 -08:00
Amjad Masad 73b032ab87 [react-packager] Update sane to get a new version of fb-watchman (perf) 2015-06-26 17:48:39 -08:00
Amjad Masad de020b44c9 [react-packager] Enable watchman fs crawl
Summary:
@public
Now that watchman perf issue was fixed we can enable watchman-based fs crawling which is faster than node.
This showed an existing issue with some files missing from the blacklist which I addressed.

Test Plan:
./fbrnios.sh run
click around and scroll all the apps
2015-06-26 16:20:50 -08:00
Amjad Masad 1109ce3b8c [react-packager] Injectible file crawlers (2x crawl speedup) 2015-06-24 18:04:57 -08:00
Jing Chen c157051da7 [events] Add JS require time and fix up some existing logs 2015-06-24 16:59:00 -08:00
Amjad Masad 4ac5c7e19e [react-packager] Fix the confused node_modules detection function
Summary:
@public

We have a function that detects whether a give file is to be treated as a node_modules. If so it doesn't have access to the haste module map. There is an exception to this rule which is a few modules that are allowed to do that. Currently thats react-native, react-tools, and parse.

The current implementation had a bug where if you had `react-native` (or react-tools etc) in the name before the actual package root then the detection will be off. This fixes the problem by starting from the `lastIndexOf('node_modules')` directory, that way nothing confuses us.

Test Plan:
./runJestTests.sh
export OSS, patch, run e2e test
2015-06-24 15:20:35 -08:00
Christopher Chedeau eb9c7235e2 [ReactNative] Bring in node console.log formatting
Summary:
@public

The current output of console.log is extremely bad. If you pass NaN, it shows up as null (super confusing I know -_-), if you pass a cyclical object, it just says that it is cyclic and that's it. It doesn't print up the first few levels which are NOT cyclical and would be really helpful.

It turns out that nodejs console.log pretty printer is really awesome and can be easily extracted as a few hundred lines. This is going to be such a productivity boost that I think it's the right tradeoff to embed it like this

Test Plan:
```
var a = {kikoo: {lol: 1}}
a.kikoo.nice = a;

console.log(a);
> { kikoo: { lol: 1, nice: [Circular] } }

console.log(NaN)
> NaN
```
2015-06-24 14:11:55 -08:00
Forbes Lindesay fccea2f365 Replace bluebird with promise 2015-06-22 08:44:03 -08:00
Bill Fisher 5aee4cec98 [ReactNative][easy] fix server 500 response typo
Summary:
@public
corrected small typo in the 500 response from the packager server

Test Plan: add throw to promise function prior to error handler, run packager, cache a bundle with bundle extension URI, open /debug/packages, see clean 500 error
2015-06-20 16:43:18 -08:00
Amjad Masad 2845e78080 [react-packager] Cache in-memory file lookups (~200ms win on file change) 2015-06-19 22:47:26 -08:00
Amjad Masad 2d4055e513 [react-packager] Rewrite dependency graph (support node_modules, speed, fix bugs etc)
Summary:
@public
Fixes #773, #1055
The resolver was getting a bit unwieldy because a lot has changed since the initial writing (porting node-haste).
This also splits up a large complex file into the following:

* Makes use of classes: Module, AssetModule, Package, and AssetModule_DEPRECATED (`image!` modules)
* DependencyGraph is lazy for everything that isn't haste modules and packages (need to read ahead of time)
* Lazy makes it fast, easier to reason about, and easier to add new loaders
* Has a centralized filesystem wrapper: fast-fs (ffs)
* ffs is async and lazy for any read operation and sync for directory/file lookup which makes it fast
* we can easily drop in different adapters for ffs to be able to build up the tree: watchman, git ls-files, etc
* use es6 for classes and easier to read promise-based code

Follow up diffs will include:
* Using new types (Module, AssetModule etc) in the rest of the codebase (currently we convert to plain object which is a bit of a hack)
* using watchman to build up the fs
* some caching at the object creation level (we are recreating Modules and Packages many times, we can cache them)
* A plugin system for loaders (e.g. @tadeuzagallo wants to add a native module loader)

Test Plan:
* ./runJestTests.sh react-packager
* ./runJestTests.sh PackagerIntegration
* Export open source and run the e2e test
* reset cache
* ./fbrnios.sh run and click around
2015-06-19 18:05:18 -08:00
Amjad Masad 0fec355076 [react-packager] Cache based on options, not url
Summary:
@public

We cached based on url, which wasn't unique becuase some options would be defaulted. This was obvious when starting the server via fbrnios which tries to warmup the bundle.
And then when the device woke up it will send a request (that is identical in reality) but would miss the cache.

This changes the cache key into a JSON stringification of the options.

Test Plan:
* ./runJestTests.sh
* ./fbrnios.sh run
2015-06-19 15:14:31 -08:00
Amjad Masad 9998337220 [react-packager] Add tests to ensure we return all dependency types 2015-06-18 21:27:55 -08:00
Amjad Masad 477360b8c9 [react-packager] Make it safe to include files without a newline at the end
Summary:
@public
Fixes #1431
Fixes #1005
Files with no newlines and a comment at the end of the file would've caused a syntax error in the bundle:

```js
__d('module', function() {
hi();
// wow })
```

This fixes the issue by inserting a new lines before `})`.

Test Plan:
* ./runJestTests.sh
* ./runJestTests.sh PackagerIntegration
* open app to the playground app
* add an error
* observe that the redbox has the correct lines
2015-06-16 12:02:49 -08:00
Tadeu Zagallo ae9e4089fc [ReactNative] Revert packager ignoring node_modules 2015-06-11 10:45:32 -08:00
Spencer Ahrens 6c6b8bec7e [ReactNative] Fix require-time redboxes 2015-06-09 15:42:47 -08:00
Eric Vicenti f90fa53df9 Revert [react-packager] Add support for nested node_modules 2015-06-05 08:44:06 -08:00
Amjad Masad 30fc7389d1 [react-packager] Fix more node_modules resolution rules
Summary:
@public
Fixes #773
This fixes `.json` name resolution. And also reads `package.json` when doing a directory module resolution.
The algorithm can be found here: https://nodejs.org/api/modules.html
I'll probably start including the node (or browserify) modules test in later diffs to make sure we're fully compliant.

Test Plan:
* ./runJestTests.sh
* ./runJestTests.sh PackagerIntegration
* open playground and require a json file
* test redbox
2015-06-04 15:07:03 -08:00
Amjad Masad 856469a24b [react-packager] Support packages with '.' in the name
Summary:
@public
Fixes issue #1055
For some historical reason we used to strip the extension of the module name before passing it to `resolveDependency` which is completly capable of handling all kinds of names. The fix is one line, but added a few tests for this.

Test Plan:
* ./runJestTests.sh
* ./runJestTests.sh PacakgerIntegration
* Open app and click around
2015-06-03 14:07:31 -08:00
Amjad Masad 5a191dadfc [react-packager] Add support for nested node_modules
Summary:
@public
The packager's resolver started out imitating node-haste, which meant that we didn't support nested modules. Now this is a problem. Bigger projects are bound to have versions of different versions of the same package at different levels of the dependency tree. This
makes loading dependencies lazy for node_modules and implements the node resolution algorithm. However, it also mantains that some
modules are still "haste" format, which currently defaults to "react-native" and "react-tools".

Finally, this means ~5 seconds speed up on every server start. This should also have a big impact on open source users with projects with big node_modules.

Test Plan:
1- test the app with --reset-cache
2- click around test and production apps
3- update the OSS library
4- create a new project
5- npm install multiple modules
6- create some version conflict in your project
7- make sure we do the "right" thing
8- test file changes to make sure it works
2015-06-03 11:35:48 -08:00
Tadeu Zagallo ebae151f24 [ReactNative] Update sane fork + FileWatcher config 2015-05-28 07:12:41 -08:00
Tadeu Zagallo 21b3304a95 [ReactNative] Add option to file watcher to ignore node_modules 2015-05-28 01:18:47 -08:00
Amjad Masad 82a082a794 [react-packager] Introduce buildPackage API 2015-05-22 17:28:27 -08:00
Amjad Masad bd11de1b51 [react-packager] Use actual error types
Summary:
@public
Previously, we had to use errors as a property on the result object because there was no way to pass custom props between
the child worker and the parent. This has been fixed in node-worker-farm (D2092153) and now we can use regular errors.
This also adapts the transformer to babel-specific errors. Generic errors, however, should still work and render readable
info.

Additionally, I deprecated, but maintained backwards compatiblity for people in OSS that are using custom transformers.

Test Plan:
1. `./runJestTests.sh`
2. `./runJestTests.sh PackagerIntegration`
3. open the playground app
4. Add a syntax error. Say `1=x` somewhere in the file
5. Reload and see error message 'SyntaxError <filename> description (line:col)'
6. Make sure that the stack trace is clickable and it attempts to open the editor to the location
2015-05-22 10:21:26 -08:00
Amjad Masad 9a76f224af [react-packager] Add first class support to popular image formats
Summary:
@public
1. Default to first class support of popular image formats
2. Add tests to make sure we support other than png

Test Plan:
1. ./runJestTests.sh
2. Add test.png and test.jpg images in the Playground app dir
3. require both images and render then in the playground app
4. they render
2015-05-20 15:28:29 -08:00
Amjad Masad 8bb65215b1 [react-packager] Implement getJSModulePaths API 2015-05-20 13:38:46 -08:00
Amjad Masad 64c0bb0bd4 [react-native] Fix source map issue with virtual modules 2015-05-15 15:49:11 -08:00
Amjad Masad 5429b5f9cc [react-packager] Use transformer name in cache name
Summary:
@public
Shouldn't confuse the cache from files transformed by different transformers. This takes into account the transformer in the cache hash name.

Test Plan:
* start server with --babel
* generate bundle
* start server with --jstransform
* generate bundle
* compare them and they're different
2015-05-13 14:49:28 -08:00
Spencer Ahrens 81ad810186 [ReactNative] differentiate fatal and soft exceptions 2015-05-13 13:24:37 -07:00
Nick Lockwood 6b2c88feec decode pathName when extracting from url 2015-05-13 13:24:36 -07:00
Dmitry Soshnikov 792b2db23c [jest] Update to v0.4.2 2015-05-13 13:24:35 -07:00
Alex Kotliarskyi c76fb40ec4 [ReactNative] Register assets with AssetRegistry 2015-05-07 17:27:42 -08:00