Summary:
This kills fastfs in favor of Jest's hasteFS. It gets rid of a ton of code, including the mocking code in ResolutionRequest which we don't need any more. Next step after this is to rewrite HasteMap, ModuleCache, Module/Package. We are getting closer to a nicer and faster world! :)
Here is what I did:
* Use Jest's HasteFS instead of fastfs. A fresh instance is received every time something changes on the FS.
* HasteFS is not shared with everything any more. Only one reference is kept in DependencyGraph and there are a few smaller functions that are passed around (getClosestPackage and dirExists). Note: `dirExists` now does fs access instead of an offline check. This sucks but stat calls aren't slow and aren't going to be a bottleneck in ResolutionRequest, I promise! When it is time to tackle a ResolutionRequest rewrite with jest-resolve, this will go away. "It gets worse before it gets better" :) The ModuleGraph equivalent does *not* do fs access and retains the previous way of doing things because we shouldn't do online fs access there.
* Add flow annotations to ResolutionRequest. This required a few tiny hacks for now because of ModuleGraph's duck typing. I'll get rid of this soon.
* Updated ModuleGraph to work with the new code, also created a mock HasteFS instance there.
* I fixed a few tiny mock issues for `fs` to make the tests work; I had to add one tiny little internal update to `dgraph._hasteFS._files` because the file watching in the tests isn't real. It is instrumented through some function calls, therefore the hasteFS instance doesn't get automatically updated. One way to solve this is to add `JestHasteMap.emit('change', …)` for testing but I didn't want to cut a Jest release just for that. #movefast
(Note: I will likely land this in 1.5 weeks from now after my vacation and I have yet to fully test all the product flows. Please give me feedback so I can make sure this is solid!)
Reviewed By: davidaurelio
Differential Revision: D4204082
fbshipit-source-id: d6dc9fcb77ac224df4554a59f0fce241c01b0512
Summary:
This removes support for `require('image!…')`, which has been deprecated for a long time.
It is still possible to use images that are already bundled by the native app using the `nativeImageSource` module.
Check http://facebook.github.io/react-native/docs/images.html for detailed documentation.
Reviewed By: matryoshcow
Differential Revision: D4231208
fbshipit-source-id: 05ec4c1ca0fabdc3fbb652f8ad1acdf240a67955
Summary: Uses `Logger` for the HMR Server output so that we get timestamps for free and the packager server output does not look as crappy.
Reviewed By: cpojer
Differential Revision: D4210827
fbshipit-source-id: e1ae64c359150b98c33e7b074f1111d37a2bc39b
Summary:
Force npm for e2e tests.
Bonus: tracked down and removed a non-standard `.done()` call that was breaking e2e tests.
Reviewed By: bestander
Differential Revision: D4153514
fbshipit-source-id: 079939874ea75c7ea6e3e5376d37873c95fc74c1
Summary:
Since jest stopped using node-haste a while ago, we are the only client left.
This brings back node-haste back to fbsource to allow us to iterate faster.
Reviewed By: bestander
Differential Revision: D3641341
fbshipit-source-id: a859f8834765723a3515e2cf265581b9dd83997c
Summary:We weren't passing `transformOptions` to `getShallowDependencies`, and therefore, when this method was called on a module, it would bust the cache and cause a retransform of the file. This was resulting in a complete retransforming of all files when the HMR Client connected to the packager.
Closes https://github.com/facebook/react-native/pull/6843
Differential Revision: D3145306
Pulled By: martinbigio
fb-gh-sync-id: 3619c27801b2fc07b758fafed47fcc892bb8e6db
fbshipit-source-id: 3619c27801b2fc07b758fafed47fcc892bb8e6db
Summary:We recently refactor the packager to transform the module names into numeric IDs but we forgot to update the HMR call site. As a consequence, HMR doesn't work the first time a file is saved but the second one.
This is affecting master as of 3/20. If we don't land this before v0.23 is cut we'll have to cherry pick it. This rev does *not* need to be picked on v0.22.
Reviewed By: bestander
Differential Revision: D3075192
fb-gh-sync-id: 410e4bf8f937c0cdb8f2b462dd36f928a24e8aa8
shipit-source-id: 410e4bf8f937c0cdb8f2b462dd36f928a24e8aa8
Summary:This brings back "Use numeric identifiers when building a bundle", previously backed out.
This version passes on the correct entry module name to code that decides transform options.
Original Description:
Since the combination of node and haste modules (and modules that can be required as both node and haste module) can lead to situations where it’s impossible to decide an unambiguous module identifier, this diff switches all module ids to integers. Each integer maps to an absolute path to a JS file on disk.
We also had a problem, where haste modules outside and inside node_modules could end up with the same module identifier.
This problem has not manifested yet, because the last definition of a module wins. It becomes a problem when writing file-based unbundle modules to disk: the same file might be written to concurrently, leading to invalid code.
Using indexed modules will also help indexed file unbundles, as we can encode module IDs as integers rather than scanning string IDs.
Reviewed By: martinbigio
Differential Revision: D2855202
fb-gh-sync-id: 9a011bc403690e1522b723e5742bef148a9efb52
shipit-source-id: 9a011bc403690e1522b723e5742bef148a9efb52
Summary:Name resolution of inversed dependencies used to happen in node-haste, but that makes it difficult switiching to numeric module IDs.
This moves the name resolution to the HMR server in order to be able to change the logic more easily.
node-haste 2.9.0 provides a `Map` of modules to `Set`s that contain the modules that depend on the key.
Reviewed By: martinbigio
Differential Revision: D3047414
fb-gh-sync-id: b98accea901d4da209dc4434ab111eb07ce9e2a0
shipit-source-id: b98accea901d4da209dc4434ab111eb07ce9e2a0
Summary:Make packager transform files before extracting their dependencies.
This allows us to extract dependencies added by transforms (and to avoid including them manually).
It also allows for better optimization and to get rid of the “whole program optimization” step:
This diff utilizes the new worker introduced in D2976677 / d94a567 – that means that minified builds inline the following variables:
- `__DEV__` → `false`
- `process.env.NODE_ENV` → `'production'`
- `Platform.OS` / `React.Platform.OS` → `'android'` / `'ios'`
and eliminates branches of conditionals with constant conditions. Dependency extraction happens only after that step, which means that production bundles don’t include any modules that are not used.
Fixes#4185
Reviewed By: martinbigio
Differential Revision: D2977169
fb-gh-sync-id: e6ce8dd29d1b49aec49b309201141f5b2709da1d
shipit-source-id: e6ce8dd29d1b49aec49b309201141f5b2709da1d
Summary:In order to be able to Hot Load Redux stores and modules that export functions, we need to build infrastructure to bubble up the HMR updates similar to how webpack does: https://webpack.github.io/docs/hot-module-replacement-with-webpack.html.
In here we introduce the minimum of this infrastructure we need to make this work. The Packager server needs to send the inverse dependencies to the HMR runtime that runs on the client so that it can bubble up the patches if they cannot be self accepted by the module that was changed.
This diff relies on https://github.com/facebook/node-haste/pull/40/files which adds support for getting the inverse dependencies.
Reviewed By: davidaurelio
Differential Revision: D2950662
fb-gh-sync-id: 26dcd4aa15da76a727026a9d7ee06e7ae4d22eaa
shipit-source-id: 26dcd4aa15da76a727026a9d7ee06e7ae4d22eaa
Summary:We've received reports saying that sometimes HRM updates take a couple of seconds to get applied. The feature is very optimized so that it takes around 100ms for most common type of changes. Only changes that require rebuilding caches could take longer than that, maybe up to 1 second.
We think the problem is that watchman delays sending the file change notification because the system is under heavy use. It worth mentioning this is not a watchman issue!. This could happen for instance if flow is enabled. So, to better understand what's going on lets log when a file is changed and just before sending the HMR update to the client. The client codepath is extremelly fast so no need to log any of that.
Reviewed By: davidaurelio
Differential Revision: D2978694
fb-gh-sync-id: abd3b473d0b7ac7cd4461effce9813ccfda32c2b
shipit-source-id: abd3b473d0b7ac7cd4461effce9813ccfda32c2b
Summary:
The bundler class had duplicated code and parts that were hard to follow, because functions accepted heterogenous arguments, leading to a lot of conditionals.
This commit tries to remove duplication between build steps of different type of bundles, and by accepting less different types of arguments. These differences are now handled further up the call stack.
public
Reviewed By: sebmarkbage
Differential Revision: D2905807
fb-gh-sync-id: ef85ea0d461a9a06a4a64480e014a5324c4ef532
Summary:
public
Introduce a header bar similar to the one shown when loading the bundle to indicate that the packager server is processing an HMR update. Hook into HMR events to show this bar when appropriate.
Reviewed By: javache
Differential Revision: D2873521
fb-gh-sync-id: a77cbb2368b75b045aa8c6ababce2f731baf514b
Summary:
public
The HMR listener needs to be invoked on the non debounced callback to avoid loosing updates if 2 files are updated within the debounce configured time.
Also, improve the time it takes to send HMR updates by avoiding rebuilding the bundle when the listener is defined. Instead just invalidate the bundles cache so that if the user reloads or disables Hot Loading the packager rebuilds the requested bundle.
Reviewed By: davidaurelio
Differential Revision: D2863141
fb-gh-sync-id: 3ab500eacbd4a2e4b63619755e5eabf8afdd1db9
Summary:
public
At the moment, when the user changes a file we end up pulling the dependencies of the entry point to build the bundle. This could take a long time if the bundle is big. To avoid it, lets introduce a new parameter to `getDependencies` to be able to avoid processing the modules recursively and reuse the resolution responseto build the bundle.
Reviewed By: davidaurelio
Differential Revision: D2862850
fb-gh-sync-id: b8ae2b811a8ae9aec5612f9655d1c762671ce730
Summary:
public
To make sourcemaps work on Hot Loading work, we'll need to be able to serve them for each module that is dynamically replaced. To do so we introduced a new parameter to the bundler, namely `entryModuleOnly` to decide whether or not to process the full dependency tree or just the module associated to the entry file. Also we need to add `//sourceMappingURL` to the HMR updates so that in case of an error the runtime retrieves the sourcemaps for the file on which an error occurred from the server.
Finally, we need to refactor a bit how we load the HMR updates into JSC. Unfortunately, if the code is eval'ed when an error is thrown, the line and column number are missing. This is a bug/missing feature in JSC. To walkaround the issue we need to eval the code on native. This adds a bit of complexity to HMR as for both platforms we'll have to have a thin module to inject code but I don't see any other alternative. when debugging this is not needed as Chrome supports sourceMappingURLs on eval'ed code
Reviewed By: javache
Differential Revision: D2841788
fb-gh-sync-id: ad9370d26894527a151cea722463e694c670227e
Summary:
public
We should further improve this on the future by showing the actual stacktrace instead of the `HMRClient` one. Also, we need to integrate this with the dev plugin that opens in the default editor the file/line the user clicks on.
Reviewed By: vjeux
Differential Revision: D2798889
fb-gh-sync-id: 2392966908c493e86e11b0d024e7b68156c9066c
Summary:
public
Fixes a terrible bug due to which when Hot Loading enabled when the user reloads we'll serve them the first `hot` bundle he requested. This happened because when HMR enabled we bailed out after sending the HMR updates and didn't rebuild any of the bundles the user requested before. As a consequence, when they reload we'd sent him the first and only one we ever built.
The fix is to tweak the hmr listener to return a promise. This way we can run the remaining code on the file change listener just after the HMR stuff finishes. We need to do it this way to avoid the remaining stuff to compete for CPU with the HMR one and give the best possible experience when HMR is enabled.
Reviewed By: davidaurelio
Differential Revision: D2811382
fb-gh-sync-id: 906932d71f35467485cf8a865a8d59f4d2ff41a0
Summary:
public
We're not planning to accept file removals in the short term on the HMR interface so lets bail when a file is removed (before this this we were throwing when trying to get the shallow dependencies).
Reviewed By: yungsters
Differential Revision: D2810534
fb-gh-sync-id: f2733382f4a2619e22bdf1163aa4180694fff9f8
Summary:
public
Requires are transformed when building the bundle but we forgot doing so when building the HMR one.
Reviewed By: vjeux
Differential Revision: D2801319
fb-gh-sync-id: ae70612945ab81a05154b14d6b756ef390770542
Summary:
public
Before this diff we were only accepting the module that was modified but the user. This works fine as long as the user doesn't modify the dependencies a module has but once he starts doing so the HMR runtime may fail when updating modules' code because they might might a few dependencies. For instance, if the user changes the `src` a `Image` has to reference an image (using the new asset system) that wasn't on the original bundle the user will get a red box. This diff addresses this by diffing the modules the app currently has with the new ones it should have and including all of them on the HMR update. Note this diffing is only done when the we realize the module that was modified changed it's dependencies so there's no additional overhead on this change.
Reviewed By: vjeux
Differential Revision: D2796325
fb-gh-sync-id: cac95f2e995310634c221bbbb09d9f3e7bc03e8d
Summary:
public
Before this this when a file was changed besides sending the HMR update we rebuild every single bundle that the packager had build (to serve it faster when the user hit cmd+r). Since when hot loading is enabled we don't do cmd+r all this work was pointless (except for when you're developing multiple apps using the same packager instance at the same time, which we can assume is very uncommon). As a consequence, the HMR update was competing with the rebundling job making HMR quite slow (i.e.: on one huge internal app it took up to 6s for the HMR changes to get applied).
So, this diff tweaks the file change listener so that we don't rebundle nor invoke the fileWatchers (use for live reload which is also useless when hot load is enabled) when hot loading is enabled. Also, it makes the HMR listener more high pri than the other listeners so that the HMR dev experience is as good as it can get.
Reviewed By: vjeux
Differential Revision: D2793827
fb-gh-sync-id: 724930db9f44974c15ad3f562910b0885e44efde
Summary:
public
Compute the dependencies of the bundle entry file just before sending HMR updates. In case the file that was changed doesn't belong to the bundle bail.
Reviewed By: vjeux
Differential Revision: D2793736
fb-gh-sync-id: f858e71b0dd5fe4f5b2307a22c6cef627eb66a22
Summary:
public
Implement all the necessary glue code for several diffs submitted before to get Hot Loading work end to end:
- Simplify `HMRClient`: we don't need to make it stateful allowing to enable and disable it because both when we enable and disable the interface we need to reload the bundle.
- On the native side we introduced a singleton to process the bundle URL. This new class might alter the url to include the `hot` attribute. I'm not 100% sure this is the best way to implement this but we cannot use `CTLSettings` for this as it's are not available on oss and I didn't want to contaminate `RCTBridge` with something specific to hot loading. Also, we could potentially use this processor for other things in the future. Please let me know if you don't like this approach or you have a better idea :).
- Use this processor to alter the default bundle URL and request a `hot` bundle when hot loading is enabled. Also make sure to enable the HMR interface when the client activates it on the dev menu.
- Add packager `hot` option.
- Include gaeron's `react-transform` on Facebook's JS transformer.
The current implementation couples a bit React Native to this feature because `react-transform-hmr` is required on `InitializeJavaScriptAppEngine`. Ideally, the packager should accept an additional list of requires and include them on the bundle among all their dependencies. Note this is not the same as the option `runBeforeMainModule` as that one only adds a require to the provided module but doesn't include all the dependencies that module amy have that the entry point doesn't. I'll address this in a follow up task to enable asap hot loading (9536142)
I had to remove 2 `.babelrc` files from `react-proxy` and `react-deep-force-update`. There's an internal task for fixing the underlaying issue to avoid doing this horrible hack (t9515889).
Reviewed By: vjeux
Differential Revision: D2790806
fb-gh-sync-id: d4b78a2acfa071d6b3accc2e6716ef5611ad4fda
Summary:
public
This diff adds infra to both the Packager and the running app to have a WebSocket based connection between them. This connection is toggled by a new dev menu item, namely `Enable/Disable Hot Loading`.
Reviewed By: vjeux
Differential Revision: D2787621
fb-gh-sync-id: d1dee769348e4830c28782e7b650d025f2b3a786
Jean Lauliac
Christoph Pojer
David Aurelio
Tim Yung
Ovidiu Viorel Iepure
Adam Miskiewicz
Martín Bigio