The Quest to the Perfect Full-Stack Framework Begins
Exploring VulcanJS, a schema-first web framework
During 2020’s lockdown weekends, after months away from web development, I took a look to the State of Javascript Frameworks, starting with VulcanJS. In this article, I will discuss what I was looking for and how various frameworks respond to this.
A schema to rule them all
My previous web app, Calendoo, a linear calendar and flight booking assistant was implemented on top of Meteor and React. Starting from Schema Validation for Remote Procedure Calls, I had progressively implemented a schema-driven system that would not only verify the types and valid contents of fields, but also determine what would be allowed for writing or what would be worth displaying in a form. So, I was enthusiastic to try solutions such as VulcanJS, a framework built on top of Meteor that claims to automatically generate GraphQL schemas for your collections, generate client-side forms and handle their submission, provide data loading helpers to make loading Apollo data easier and handle security and validation for you, while providing a modern React+GraphQL stack supporting page hot reload, Server-Side Rendering and many other features out of the box.
A Newbie Journey
Discovering a new framework can be overwhelming, so framework developers often put a lot of effort in providing an interesting, if not fun, tutorial.
This is the case with VulcanJS: the tutorial showcases the capabilities of the framework in a brilliant way thanks to hot reload: the instructions are fully included within the tutorial app. Also, the Vulcan-Starter repo not only contains the tutorial app but a few other demos, so you can browse common scenarios and see patterns to reuse later.
Going off tracks
Now that you have completed the tutorial, you are ready to build your own app by adding a new package, while commenting the previous ones. But then, you’ll want to dig the documentation to find how to achieve a certain outcome. This is where I encountered some hiccups: the documentation was lagging behind the codebase, and I had to spend time in tracing internal of the framework to find out how to actually use it, and contact the authors on Slack to ask for guidance. Then I provided a pull request of the documentation, but since the authors were in the process of evolving the API, some proposals were not matching their direction.
A great framework should provide one documentation tree per major version (see the Meteor docs as an example), including work-in-progress drafts. It must also let you know clearly which version you are currently using.
Getting future-proof
When you are evaluating a framework, you’re thinking ahead for many years. You are not going to stick at the version you pulled initially, are you? So testing the upgrade process is paramount. This test includes not only the library upgrade itself but also the action you need to take to upgrade your own code and data.
In the case of VulcanJS, I had a local Git copy of Vulcan-Starter but I had swapped the .git folder in order to commit my app to my own repository. So I had to swap it back and pull the next major version tag. I also did it in the separate VulcanJS repo that you can link from your app as part of the optional two-repo install. Overall, except some package.lock subtleties, it did not pose any issues. I also figured out which versions constants to modify.
But then, I did not find any guidance in the doc for modifying the code, apart from fixing issues at they appear. The Vulcan authors are now working on providing a migration document, and at the time they also wrote a blog post to explain any breaking changes. Upgrading from 1.14.1 to 1.15, I didn’t have a lot of issues, but any bigger upgrade would become problematic if not following an upgrade recipe or human-readable Changelog.
Make sure that your framework provides instructions for upgrading chores. Even with clearly documented upgrades such as in Meteor, based on my experience, things can turn bad quickly.
Especially if your framework does not benefit from a large user base, which means you won’t find a lot of ready-made answers on the Internet.
To avoid tedious long and error-prone manual testing sessions after each upgrade, you may want to write some automated tests. That’s quite easy to do on the backend API, but what about the UX? In the past, I’ve used Nightwatch. Since human interaction is quite complex and details matters, writing scripts detecting every little UI hiccup can become very complex and time-consuming. I don’t have a perfect solution here. The future Next.js version of Vulcan is using Cypress and tries to catch regressions at framework level. Sounds promising!
Size matters
I also tested another feature that is important for quick app startup after your codebase grows: downloading scripts on demand client-side. I had already used the dynamic import feature in Meteor, and was pleased to find it as well in Vulcan.
Try to think what will happen, in terms of building and distributing the app, when your codebase reaches several thousands of lines of code.
I don’t remember having any issues with build times back in 2017 on Calendoo, but it seems 2020-Meteor project, with a lot of new integrated frameworks, such as Apollo GraphQL client, takes some time to rebuild, which makes the time to hot reload your Vulcan page quite noticeable during iteration cycles. Vulcan is transitioning to Next.js to take advantage of advanced features of Webpack to support larger codebases and quicker iteration, while Meteor contributors are also working in this area. For a start, you can now disable some build targets, such as legacy browsers of mobile during rebuilds.
I come from a game programming background, and, even if we’ve made progress to modularize game engines, you still end up working with multi-million-C++-line codebases. Which gives you some knowledge about scaling up the codebase, blobbing distributing builds and so on, but I digress!
What you win and what you loose
It’s always rewarding to try new technology, because you get to learn new ways of doing this. For instance, the VulcanJS tutorial taught me how to write functional components in React and I took the opportunity to write my small app using React Hooks.
However, mature frameworks tend to attract more features, since a long time and a large user base gives a chance for some dude to come with something the authors wouldn’t have thought of or wouldn’t have time or will to provide in the core.
Concurrent multiuser behavior
In this case, since Vulcan is built on top of Meteor, I was assuming that it inherited the powerful reactivity features of Meteor. But then, in my multi-user app, when I was modifying the record on one user’s side, it did not seem to update on the other side! And it turns out that Vulcan does not provide (yet?) additional hooks on top of Apollo to push changes to client listeners using Apollo Subscriptions, unlike Meteor’s default Pub/Sub system based on MongoDB and the DDP protocol. So, in Vulcan, either you provide user controls for refreshing the state, or you wait for the framework to poll a new version of the doc, which currently takes up to 20 seconds by default (but can be customized).
This can be perfectly fine if your use case supports it. Also, it may, in theory, allow to scale up using less ressources that those required by Meteor. Anyway, supporting a very large number of concurrent users often requires refactoring the code and using a queuing system, plus specific optimizations for your own use case.
Bottom line: when you read about the benefits of a framework, try also to scratch under the surface to discover what you won’t get, compared to alternatives. Estimate your needs in terms of future concurrent users.
Rich ecosystem
Another example of the benefits of a popular component framework is the easiness to integrate ready-made third-party components.
In my little proof-of-concept app, I was able to integrate components for QR Code generation and for QR scanning in a matter of minutes, thanks to the popularity of React.
A benchmark is set
Now that I’ve seen what I can expect from a modern framework, I will try another one, or maybe a couple of others. My experience gave me ideas for new criteria.
Catch errors earlier
Apart from the documentation inconsistency, my productivity was impacted by stupid Javascript mistakes that were not catched until the page was reloaded in the browser, such as writing:
import Symbol from ...
// instead of
import { Symbol } from ...
// when Symbol is not exported as defaultor
const MyComponent = (document, currentUser) => (…)
// instead of
const MyComponent = ({document, currentUser}) => (…)Having written mostly Golang, C++ and C# (compiled languages) in the last few months, I found this trap very annoying: not only the feedback takes several seconds, but the error is often not obviously reported.
In my next evaluation, I really want to try Typescript, even if I don’t know yet if it solves the include issue. In Golang, there are no such tricky import issues, since any symbol starting with a capital letter is considered exported and available externally! Vulcan is also transitioning to Typescript.
But I may try out Titan which also gathers Meteor, Typescript, React and GraphQL in a box. Not sure whether Server-Side Rendering will be as straightforward as with Vulcan, though.
What-You-See-Is-What-You-Get in the debugger
High-order Components (HoC) are now very popular in component-based UX programming. Vulcan uses them by default to provide data coming from the server.
As a result, when you look at your component tree in the Chrome debugger, you can see an insane depth of components!
There’s nothing wrong about it per se, but I can’t help but thinking that the bloat probably reduces performance and increase latency, even though performance in a Production environment is much more optimized than in the debugging environment.
I will keep an eye on that on my next framework evaluations. It would be worth trying Svelte, which claims to produce thin and performant UX code.
Schema-based
This is clearly the most interesting part of Vulcan.
I would like to try Uniforms that claims to generate client-side forms from a simple schema. After all, using it with a mature framework might prove a great compromise!
To be continued
Stay tuned to hear about next trials.
Now, I surely haven’t covered all criteria here. What are your must-have and good-to-have?
Opinions in this article are my own and not the views of my employer.
