Remixing zCitations: Modernizing a Legacy Web App with Remix and Carbon
Introduction
When I joined the Content Design & Development (CDD) team in IBM Z in 2019, my primary responsibility was to maintain and enhance two web-based applications used by the z/OS documentation authors. Both apps had already been around for over a decade at that point. The most important of the two was called zCitations, a tool that was used to manage many different kinds of book metadata and make it easier to build DITA source into HTML and PDF. The other app, ID Service, was used to create RTC Tasks to process documentation changes for closed APARs.
For four years, I gradually added functionality and fixed defects in these apps. But in 2022, it had become very clear that zCitations needed a major overhaul. This article describes our team’s path to reinvent the app from the ground up, using the cutting-edge combination of Remix and Carbon.
We’ve Got Problems
The journey described herein began primarily out of a desire to make zCitations more secure with regards to the third-party libraries being used. zCitations was written in a mix of mostly PHP, some JavaScript, and HTML with embedded PHP. It was implemented with the CodeIgniter Release 3 framework (CI3), and thus used many proprietary CI3 libraries. It also employed a substantial number of third-party libraries in both PHP and JavaScript, many of which were quite old and out of service, so simply upgrading those modules was impossible. As a result, a security scan of the source code revealed many security vulnerabilities that were effectively unfixable without a major change of some kind. While zCitations was an internal-only app and thus the need to fix the vulnerabilities was not extremely urgent, the scan opened our eyes to just how old all of our dependencies were.
Of secondary concern was the UI appearance and behavior. Because it had been built over many years with a mish-mash of different third-party JavaScript libraries by many developers, two pages in zCitations that both used a table to display data could look and act completely different. Problems of this sort could be rectified by redesigning a subset of the functionality so that all pages used the same UI libraries, but even that work alone would have been a sizable endeavor. If substantial time was going to be invested into UI rework, perhaps it would make more sense to rethink the entire app design and truly modernize it?
Shown below in Image #1 is the Continuous Delivery page from zCitations. Note the messy button layout and the lack of row detail expansions.
Phase 1: CodeIgniter 4
The most obvious solution seemed to be to upgrade from CI3 to CI4. When our workload allowed more investigation, we opted to try it out with our ID Service app. Written for the same CI3 framework but with a tiny fraction of the lines of code in zCitations, it allowed us to more quickly and easily determine how big a job migrating to CI4 would be. Many of the same vulnerabilities found with the zCitations scan existed in ID Service as well, due to the similarities in dependencies. So if the CI4 migration mitigated the risks dramatically for ID Service, we could naturally expect similar improvements for zCitations.
The official CI3-to-CI4 migration documentation was daunting. The two releases were fundamentally different at the design level and it was clear from even a brief reading that the changes would be both time-consuming and complicated. While it wouldn’t require a complete rewrite, it would involve multiple person weeks of coding effort and a great deal of retesting.
And so it did, but in the end the effort was only partially successful: while the new implementation worked well and was put into production, moving to CI4 did not eliminate the security concerns. Many of the vulnerabilities were still out of our control to remediate. Putting unknown months of effort into migrating zCitations to CI4 would not provide the magic bullet we were after, so we decided to abandon that idea.
Phase 2: Angular with Material
The obvious next step was to invest time into a 100% rewrite and move completely off of CodeIgniter and PHP in favor of a more modern web framework. While this would involve considerably more time and effort, using a cutting-edge framework would future-proof our apps, allow for both front-end and back-end redesign, and greatly improve the maintainability of the codebase.
The question was: what framework should we use? JavaScript was the obvious choice for the language itself, but there were many popular JavaScript-based frameworks to select from: Angular, React, Vue, and many others. Since the team already had some experience with Angular and Material UI, we opted to do a simple proof of concept (PoC) with that framework first. We opted to implement our Continuous Delivery functionality first, since it was one of the most powerful and important pages in zCitations, and would also benefit the most from redesign. The PoC was just a read-only version of the page, but still visually illustrated everything that the page would ultimately provide.
In a matter of a few weeks we had a basic working demo that we could show off to our users. The app was tentatively renamed to “Content Manager”. However, user feedback convinced us that the UI design, while promising, would be better implemented in IBM’s own Carbon UI rather than Material. This decision would sow the seed for our movement away from not only Material, but Angular as well.
Shown below in Image #2 is the Continuous Delivery page from the Angular implementation. While this was much more feature-rich than the CI3 implementation, it bears very little resemblance to other IBM web sites.
Phase 3: React with Carbon
Carbon supports three JavaScript frameworks: React, Angular, and Vue. This prompted research into these three frameworks from a higher-level perspective: which of the three had been adopted the most by the industry at large? Which had the strongest upward trend in usage? Which had the most answered questions on Stack Exchange? Which had the highest likelihood of still being used five, ten, or even twenty years from now? Our findings suggested that the answer to all of these questions was the same: React. Carbon also happens to be natively developed for React, which is then ported to Angular and Vue. So in every conceivable way, React seemed to be the best choice for our team.
And so, a second PoC was developed with vanilla React and the Carbon UI libraries. This required considerable education in both technologies for our team, but we found the transition from Angular to React relatively straightforward, and it actually resulted in simpler, easier-to-understand code. By using Carbon, suddenly the app looked and behaved like virtually every other IBM web site. Carbon provided much more powerful, consistent, and intuitive UI functionality compared to anything we had used before. Users found it easy to learn how to use the new CDD Hub PoC UI since the controls were already familiar to them (from having seen similar implementations on other IBM web sites).
This was all very positive progress, and further demos resulted in much more encouraging feedback. We knew we were finally on the right track.
We also encouraged our users to come up with a new name for the app, since “Content Manager” was too vague and generic. The name “CDD Hub” was suggested, which has since been adopted.
Shown below in Image #3 is the Continuous Delivery page from the React/Carbon implementation, with elegant space-saving side navigation (text descriptions for each icon appear when the nav is hover-expanded) and even more features than the Angular PoC. And since this is now “Carbonized”, it offers the same accessibility-friendly behaviors and corporate look-and-feel IBMers have come to expect.
Full-Stack Decisions
At this point we were fairly certain that React and Carbon would be part of the final solution, and the name of the new app had been decided. We were almost ready to start work on redesigning and reimplementing all of the zCitations and ID Service functionality. However, one large problem remained. Carbon and React solved the front-end side of the equation beautifully, but our app required a back-end before we could move forward. This was mostly because of our database needs, but also to handle SSO more smoothly and robustly on the server side. But our PoC had no true server yet.
So, back to the research drawing board. What back-end or full-stack solutions would work well with our database and the choice we’d made for our front-end? Many companies, including IBM itself, used Next.js for their React-based full-stack solutions, so we did some reading about that option at first. But after searching for similar technologies, we discovered this article:
https://remix.run/blog/remix-vs-next
Enter Remix. While the framework was still very new at the time (release 1.x), the points made in the article convinced us that this bleeding-edge technology was worth the risk. Given its basis on (and association with) React Router, a very stable and extremely commonly-used library, we felt that we were betting on tech that would eventually surpass Next.js in industry adoption.
One simple sentence from the article was highly compelling to us: “The Remix apps are completely done before the Next.js app even starts loading images.” One of the issues we always had with our CI3 implementation was sluggish page loading performance. There were many reasons for this, such as an overly complex and inefficient design and implementation, reliance on old and slow third-party libraries, a mish-mash of different UI libraries poorly suited to the problem sets, and forcing the user to wait synchronously for some long-running tasks that should have been asynchronous back-end jobs. The zCitations design made fixing many of these problems very risky and difficult. So one of the primary benefits we sought from a new framework was speed. If Remix was basically blowing the doors off of Next.js, we definitely wanted Remix.
Phase 4: Remix with Carbon
Once again, much education was required; none of us had even heard of Remix before. There was also a lot of integration work to be done to get everything working end-to-end for a robust PoC: from UI mouse clicks to database retrieval and storage, with change validation on both client and server ends. This is where Remix made life much easier than our PHP/CodeIgniter solution. Not only was the client and server code now able to be 100% JavaScript/JSX, the exact same validation functions could be called at runtime on both sides of the wire; before we had to duplicate the same logic in PHP and JavaScript. And separating client-side-only from server-side-only code is made trivial by simply using the word “client” or “server” in the source filenames. Also imagine a back-end error, such as a database insertion fault. Instead of manually capturing the problem and converting it into something the front-end can display, we simply use Remix’s ErrorBoundary handler to consistently deal with errors the exact same way across the entire app. The list of benefits to the developer goes on and on.
Shown below in Image #4 is the Continuous Delivery page from the Remix/Carbon implementation. While this screenshot naturally looks almost identical to the React/Carbon version, there is much more refinement and power now that we have a seamlessly integrated back-end responding to actions. This is no longer just a static web page, as in all previous incarnations, but an active application that is auto-refreshing table data asynchronously in real time as changes occur in the database.
Once this new Continuous Delivery page was fully implemented and production-ready, we made the equivalent page in zCitations auto-redirect to CDD Hub. This made transitioning to the new app easy: users could continue to use zCitations as their start page, but mouse clicks on migrated functionality would automatically take them to the new app in a separate browser tab. CDD Hub utilized the exact same database, so no db migration was needed, and the two apps could work simultaneously from the same live data. Over time, one page at a time, the rest of zCitations was gradually migrated to the new app. This approach allowed users to learn how to use the new app one function at a time. And because all pages now look and function identically in the new app, once a user understands how to use any one page, they essentially know how to use the entire app. This was not the case with zCitations, where virtually every page used a different UI library, each with their own idiosyncrasies.
Full Production
Today, all functionality from both zCitations and ID Service has been successfully moved into CDD Hub and deployed onto a production VM on OneCloud, running 24x7. The old zCitations and ID Service apps have been disabled and archived. CDD Hub is not only used by the z/OS content authors, but has also been adopted by z/VM and z/Hardware, and also offers a feedback page to allow z/OS software developers to create new work items for z/OS documentation updates.
The total time it took a single developer to rewrite both apps, including the Remix and Carbon education, was only nine months. Once the Remix and Carbon basics have been learned, it’s astonishing how quickly new functionality can be written and deployed. And one of the primary motivating factors from the very beginning, which was to reduce or eliminate security vulnerabilities, has been achieved:
Image #5: security audit
Final Thoughts
The new app has incredible benefits for developers and users alike, thanks to Remix and Carbon:
- Modernized look and feel with cutting-edge technology
- Much faster page loading and UI response
- Better design: much more intuitive to use
- High-quality, reliable, predictable code with extremely low defect incidence
- Enhancements and fixes now take hours instead of days
- Extremely easy to enhance and maintain
- Much more secure and much easier to apply patches
- More accessible thanks to Carbon
- Wealth of new back-end automation features and administration function
From a personal perspective, the Remix framework has completely transformed this author’s career from one of simple enjoyment to one of absolute passion. In over twenty-five years of software development, never before has coding been so thoroughly enjoyable and rewarding. If you want to fall in love with your job, write your next PoC with Remix and Carbon!
Ernest Crvich is a Software Engineer at IBM based in Boulder, CO. The above article is personal and does not necessarily represent IBM’s positions, strategies or opinions.
