I have yet to meet a serious frontend programmer who hasn't used file
system watcher tasks to recompile/rebuild/redeploy code and enjoy the
instant feedback. In Javascript land all build tools/bundlers/packers
and whatnot provide watch​ flags to make them watch for changes
forever. This sounds a good idea at first, but when you have to use
more of these tools in the same process, it gets unmanageable. But why
do we even need these watchers? Transpiling, bundling and minifying
Javascript takes a non-negligible time and is a rather complex process.
Eg., bundlers have to create a module graph to resolve dependencies
between source files. Browserify as well as Rollup uses an in-memory
cache for these and can detect changes in the module graph between
successive builds so it can recompile only the necessary parts. This is
known as incremental compilation and makes building significantly faster
for small, local changes. And we know that programmers usually make
exactly these kind of changes during development. If you caught the
'in-memory' qualifier, you are now also aware of the limitation of
these tools, ie. they only retain the cache within a single process,
hence the endless watch tasks. The problem with watchers is that they
don't fit with the well-established, cosy, (mostly) correct and fast
model that traditional build systems like
Make and its derivatives use for
incremental building. In these you declaratively define targets with
requirements, and let the build tool figure out changes and rerun the
necessary steps to update the targets in the task graph (directed
acyclic graph). This is great because if you have well defined
dependencies everywhere, the build system ensures that you have correct
and fast incremental builds all the time. It is also great because the
build tools is no longer a blackbox, and you and only you define the
requirements for your targets. But what if defining the exact
requirements is too complex, and you simply have to delegate it to
Browserify, Rollup or whatever? You
cannot put an arbitrary watch target into a makefile as it will run
forever, so it has to be a top-level one! If you have different such
watchers, coalescing them into a top level task quickly becomes a
nightmare. Gulp aimed to solve this problem with
its 'streams instead of files' philosophy, which is nice but who the
heck wants to write build files in Javascript ever? I so loved
rebuilding my thesis written in LaTeX with a simple make target
using fswatch - a cross
platform command line utility for watching for file system changes. It
is the only watcher I am willing to use!
watch :
@echo Stop watching with Ctrl-C
@sleep 1 # Wait a bit, so users can read
@$(MAKE) || exit 0;
@trap exit SIGINT; fswatch -o $(INPUT_FILES) | while read; do $(MAKE); done
.PHONY : watch
So I hacked around the bit and I figured out that with Rollup, I can actually save the cache, load it later and it just works. Let me show my complicated build file of my toy project here. I have two recipes:
- one that calls a
compile-mappings.js script that just assembles a lovely XML (no smirking!) that I happen to know that exact dependencies of, - and a
compile-scripts.js that uses Rollup for bundling my source files into a fat JAR... erhm sorry, bundle.
I want to watch both of them, so I cannot stall make just to run
rollup --watch for sake of the second task! I instead do the following
in compile-scripts.js:
readFile('tmp/cache.json')
.then((cache) => JSON.parse(cache))
.catch((err) => null)
.then((cache) => rollup.rollup({ cache, entry, plugins}))
.then((bundle) => {
const cache = JSON.stringify(bundle)
return mkdirp('tmp')
.then(() => writeFile('tmp/cache.json', cache))
.then(() => bundle.write({ dest }))
})
I am trying to use a previously saved cache and creating a new one when bundling is done. And it works! This way I can use my regular make recipes and just run them manually to enjoy the shorter build times, or create the same top-level watch task for them already shown above. This solution might not be as fast as an in-memory cache, but I am glad that it works and plays nice with my existing build tools. So guess who's watching?
 Â