Bundling Application code for dist versions has always been a challenge, especially in case you want to share code with several Apps or load multiple Apps on one Page.
- The “data flows downwards” paradigm
- How does importing JS modules directly into the Browser work?
- Is the Worker Scope supported as well?
- How can we achieve the same behaviour for our dist versions?
- A Webpack Love Story
- What is the tradeoff?
- Introducing the neo.mjs v1.4 Release
- Can we achieve the same without using neo.mjs?
- What can we do with this now?
- What is coming next?
- What is happening on the neo.mjs related job market?
This article is not going to cover what “Micro Frontends” are in detail. There are very good blog posts out there on this topic, just google the term in case you have not heard about it yet. An in depth knowledge is not required to follow this article.
What we are going to talk about are the main problems which made the Micro Frontends topic popular in the first place:
- How to share code (modules) across Apps
- How to load multiple Apps on one page with close to 0 overhead
- How to bundle SharedWorkers driven multi Browser Window Apps
In case you have been following my recent blog posts, you have most likely seen the multi Browser Window Covid App:
When I started using SharedWorkers, the first Demo Apps did run inside the dist modes (including Firefox). However, Apps using multiple main threads did not. With the “cross Apps split chunks” concept, this is now possible and this is definitely a major breakthrough. Since this concept can solve a lot of client side architecture related problems, I would like to share my knowledge with you.
2. The “data flows downwards” paradigm
Like in real life, parents are very much aware of their children and have (in theory) the ability to change them. Children however have not really a clue who their parents are and are not allowed to do so.
When it comes to creating modular code, it is super important to stick to this idea. Components can adjust their child Components as they like, and it is fine to directly call methods on child Components.
Since we want to use child Components in different spots, they are not allowed to touch their parents. Instead, child Components fire events, which parent Components can listen to.
While creating the neo.mjs framework, I am trying to stick to this design pattern as good as possible. I can strongly recommend to do the same.
3. How does importing JS modules directly into the Browser work?
Modern Browsers support using JS modules directly. Take a look at:
The static import statement is used to import read only live bindings which are exported by another module. Imported…
Inside your JS files, you can use static imports to fetch the dependencies and you can use dynamic imports to (optionally) lazy load other modules.
The W3C specs are really nice and Browsers do a great job at caching JS modules. An easy example: You create Component2 & Component3 classes (inside modules), which both extend Component1. The Component1 module will only get loaded once.
We definitely want to achieve the same behaviour inside our dist versions!
I recently stumbled upon the caching with the neo.mjs entrypoints for Apps. So far they looked like this:
While this works perfectly fine for Single Page Apps, there is a catch when it comes to multi Browser Window Apps, where a SharedWorker fetches multiple Apps.
The first App defines Neo.onStart() which was executed after the lazy loading was done. Then you load a second App to show it (e.g. inside a new Window) and Neo.onStart() got overridden. The new method got triggered and this worked fine as well. Do a reload on the first App (Browser Window), the app.mjs module was cached, Neo.onStart() was not overridden and the framework loaded the wrong App:
It was easy to fix of course. The new entry points look like this:
We just export the method => the cached module will return the correct method.
4. Is the Worker Scope supported as well?
When looking at the bottom of the MDN article, you will notice that using JS modules inside main threads has a very good support, while using JS modules inside Workers is still only supported in Chromium.
This is still the reason for the neo.mjs dev mode (no builds or transpilations) to only be able to run in Chrome and now Edge. Keep in mind that your App logic runs inside the App Worker.
Webkit and Firefox have tickets for supporting JS modules inside the Worker scope (which is included in the W3C specs). We should keep an eye on those. It would be nice in case you add comments there, since it is an important topic for the future of UI development.
(There is a patch added inside the last comment, so there is hope)
1247687 — Implement worker modules
NEW (nobody) in Core — DOM: Workers. Last updated 2020–07–14.
5. How can we achieve the same behaviour for our dist versions?
So far the build entry points for neo.mjs Apps looked like this:
Each App dist version was a combination of the App Worker and your real App code base. This did not only act differently than the dev mode (where the App Worker lazy loads Apps), but was also very limiting when it comes to sharing code (split chunks).
Obviously, we can import modules from e.g. a shared folder and they do end up inside the build, but this approach made it impossible to load multiple dist versions of Apps on one page.
You could of course create multiple Apps inside the app.mjs files to get them into a build at the same time, but what we really want is to separate the build outputs for multiple Apps from the App Worker and also to get split chunks across multiple Apps.
In case we load App1 into the App Worker and at a later point want to lazy load App2, we want the dist versions to not include duplicate code (modules). This is not only bad from a file size perspective, but it can create bugs (imagine the IdGenerator does get multiple versions).
The required changes where on my todo list for quite a while, now I have finally found the time to wrap them up.
6. A Webpack Love Story
I was actually not sure if this concept is too far away from the original Webpack Scope (bundling Apps) and if this was possible at all.
We want to bundle a framework as well as multiple Apps inside a Worker Scope, including split chunks for pretty much everything.
With a bit more thought and time, it worked out surprisingly well. So, at this point I need to say it:
A big Kudos to the Webpack Team for creating such an amazing product!
Ok, now it is time to get technical :)
To visualise what we want to bundle:
Our App related index.html files look like this:
It is very important to look at the bottom => line 22. Each index file will only include the neo.mjs Main Thread. This one does support optional main thread addons for quite a while now (those are already bundled using split chunks)
Main.mjs will create the Workers (App, Data & Vdom) for us. As soon as the Workers are ready, the App Worker will fetch the app.mjs file of your App (as mentioned in Section 3).
Let us take a closer look at the new logic to do this:
The dynamic import has to run directly inside the Browser. It does need to work for the dist versions as well (Webpack). Webpack can not be aware of our custom path(s), so I am using the strategy to import literally all /app.mjs files.
To keep it reasonable, we do not want to include node_modules. I might need to polish the webpackExclude a bit more (still needs testing). It is supposed to do the following:
- When building the App Worker for the neo.mjs framework itself, we want split chunks for all possible Apps inside the apps and the examples folder
- When using neo.mjs as a node module, we only want split chunks for our own Apps (and not the examples & apps included inside the neo.mjs node_module). This part still needs testing.
- When building the Online Examples, we trigger the build on the neo.mjs node_module, in which case we do want to get the content (This part works fine).
Talking about the various buildScripts in detail would go off topic and convert this article into a book. However, this is the beauty of Open Source: You are very much welcome to dive into the code base on your own.
In case we run a dist/development build for the App Thread, the build time takes 3.148s on my machine.
The screenshot only shows a fraction of the chunks. The chunks are pretty big inside this mode, but we rarely need it anyway (sticking to the real dev mode & dist/prod).
In case we open the Covid App (non SharedWorkers) inside the Browser, we now get:
The important part is the console (dev tools). You can see the central App Worker chunk and split chunks across our different Demo Apps.
Let us do a dist/production build next:
The prod build is a bit faster: 2.191s on my machine.
The chunk file sizes are way smaller now. We do get numeric chunk names (I am mapping them into a chunks folder to keep the top level clean), which reduces the mapping overhead as well as tree shaking across our Apps.
Looking at it inside the Browser:
You will notice the numeric names now.
7. What is the tradeoff?
Like all things in life, changes come at a cost.
The old build strategy allowed us to parse a single entry point, so the build speed was faster.
Now, when we want to build one of our Apps, we need to parse all Apps inside the workspace to get the split chunks.
The build times are still very reasonable, but I polished this concept a bit more.
Running buildThreads => app
will parse all /app.mjs files and create all index.html files.
Running buildMyApps => Covid
will still parse all /app.mjs files, but at least only re-generate the index file of your selected App(s).
Time wise, it does make not much of a difference though: 2.123s
8. Introducing the neo.mjs v1.4 Release
My original plan was to finish the Calendar implementation for v1.4, but the new build processes felt more important and are definitely worth a minor release on their own. The Calendar is pushed into v1.5 now. The drag&drop implementation is already in place.
With the Cross Apps Split Chunks, we can now run our SharedWorkers driven Apps inside the dist modes.
This is the Multi Window Covid App in Firefox. Please keep in mind that Webkit / Safari does still not support SharedWorkers at all. The related ticket is here:
149850 - Reinstate support for SharedWorkers
Bug 149850: Reinstate support for SharedWorkers
Please do add some weight on it. It would be sad to see Safari becoming the next IE6 and all iOS related Browsers have to use it.
This article was not just theory-crafting. The Online Examples are updated already and all of them are using Cross App Split chunks already:
You are very welcome to dive into them (Desktop) and check the sources inside the App Worker Scope on your own:
9. Can we achieve the same without using neo.mjs?
You will need to create a central entry point for your Apps & lazy load the real code to give Webpack a chance to figure out the Cross Apps Split Chunks.
You will need a webpackInclude regex pattern to fetch the Apps for which you want to create Split Chunks, but this is it already.
10. What can we do with this now?
In case you happen to have use Ext JS in the past, you are probably familiar with the packages concept. In case you want to share code across Apps, you needed to put this code into specific packages which then could be compiled & lazy loaded (oh gosh, this was very slow).
With the new neo.mjs frameworks, you really don’t need to think about sharing & re-using modules inside different Apps anymore.
You can just load multiple Apps (dynamically) into 1 page and with the split chunks in place, there will be close to 0 overhead. This is why I mentioned “Micro Frontends” as a comparison.
A framework should be flexible & allow you to create Application architectures as you like. This is possible now.
However, I strongly recommend to stick to common design patterns. While you can define a module inside App1 and directly import it into App2 inside the same workspace, it feels a lot cleaner to put shared module code into its own spot (to make it clear that it is used inside multiple spots).
11. What is coming next?
Well, once Browsers are ready to support JS modules inside the Worker Scope on their own, I definitely want to create a new build mode without bundling. Just minifying each module file on its own. While this is not perfect for tree shaking, it would feel more the same compared to the dev mode. It does not really make sense before Browsers are ready, since we do not want to re-write the Harmony imports logic.
On the short term: As mentioned, the Calendar implementation is still at the top of my todo list. We got feature requests for menus, toast messages, a buffered grid (this is a huge one). There is work on the core level left as well: improving the delta updates performance and creating benchmarks for the config system (e.g. the cloning part).
With the drag&drop logic in place, I definitely want to create a demo where we can drag in app dialogs from one Browser Window into another one:
This will get pretty exciting for Multi Screen Apps in general and is definitely worth a new blog post once ready.
12. What is happening on the neo.mjs related job market?
I am very excited to announce that there are some big client projects starting right now which are based on using neo.mjs.
It is still my main goal to set up a Professional Services team soon, which can help clients on their way to successfully get neo.mjs Apps into prod.
I got asked “Tobi, can you recommend me for client projects?” recently quite a lot. Clients will ask: “What have you done using neo.mjs so far?”.
Right now, there are still very few developers who are up to speed. The demand for neo.mjs devs (the market in general) is growing.
The early bird wins. I can just strongly recommend you to dive into it and get up to speed.
You are more than welcome to contribute to the code base:
neo.mjs enables you to create scalable & high performant Apps using more than just one CPU, without the need to take…
You could just create an own portfolio (Demo Apps, custom components) as well. Completely up to you.
Without having any neo.mjs related code out there, it is simply impossible for me to recommend you for client projects.
Best regards & happy coding,