[Hackathon Mini Series] — Component Version Comparison
How we tackled and solved a problem from 2 sides in 2 days.
Welcome to our sixth blog of our mini series on our Hack-to-the-future hackathon. This time we have Clare talking about how one team solved a problem that had been bothering their customers for some time, all in 2 days!
About a month ago, a group of people from across Vodafone came together for our Hack to the Future hackathon — this was a 2 day event where we would come together and work tirelessly (in amongst the pizza breaks) to create something innovative. We were challenged to come up with projects that inspired us and recruit people to our teams to work on them over the 2 days.
The project I was part of was to increase visibility of what components were being used across our internal CMS (Content Management System) platform, and to highlight any major differences that we could then use to prioritise version bumping, or that would allow our internal customers to see what we were really using without having to ask us why they couldn’t see the latest features in their pages.
This will make more sense when I explain some more of the background and the problem statement in more detail.
Our Team
There were 7 of us in the team, a mixture of developers, architects, and UX and it was a great opportunity to work with colleagues we perhaps hadn’t met before in person or hadn’t worked with before.
Some Background
Before we get into the detail, it’s useful to give some background so you understand what we are talking about.
Firstly, in Vodafone Digital we are using a library of web components called Source Web.
Vodafone teams can create websites and applications from reusable, accessible, and composable React component packages by using the library.
The Source Web library gets a lot of updates and enhancements every week and therefore where possible it’s important, and beneficial, to keep up with the latest developments to take advantage of any enhanced features that are being added. Remember this for later.
Source Web has an online (internal) documentation site (docsite) where designers, developers, and anyone who is interested, can see the details of a component, it’s props, and any other relevant information.
This brings us on to Contentful.
Contentful is a highly scalable and customisable headless CMS that we are using in Vodafone UK and are in the process of migrating all our marketing and shop content over to it.
There’s a lot going on behind the scenes on how we are utilising the platform, but for the purposes of this article, the main things you need to know are:
- content editors add their content into the Contentful UI. This content is added into pre-defined content models to determine what fields need to be filled in
- we take that data using Contentful’s delivery and preview apis and pass it through some mapping functions
- referencing the content model that was used to create the content, these mapping functions transform the data into props that we can either pass straight into the relevant Source Web component, or that we can use to compose several Source Web components together to form what you see on the page
- we can then render the page or content either on demand or via SSR
A key point to note here is that when we map into components, we are mapping to a specific version of those Source Web components — or at least a specific major version in most cases.
And a final thing on this Contentful summary; remember I said at the top of this section that Contentful is highly customisable? One really great feature of the platform is the ability to extend it with your own apps which can hook into the data and enhance the content editing experience.
A quick note for clarity
To make it easier when talking about our Vodafone Contentful implementation, I will just refer to it as Contentful, but please remember when I do that, I am meaning our platform where we add data to Contentful, we map into components, we render content, and so on. It’s just easier if we refer to that as Contentful for the rest of the article. OK? Great.
Problem Statement
So, onto the why we wanted to work on this project for a hackathon.
- Bugs are raised due to functionality not working as expected so time wasted on investigating issues caused by component version differences between our mapped versions and Source Web latest versions
- Lack of clarity on which components are available in Contentful for both existing and potential users of the UK CMS platform
- Lack of visibility about component versions for our development team which also impacts prioritisation of our backlog
Initial plan
We were lucky enough to have experts from the Source Web library and Contentful in our team so after an initial brainstorm of ideas, we spit into into 2 sub teams.
This meant that we could all play to our skills, work quickly, and share our knowledge with the other sub team — one of the benefits of a hackathon is to learn something new, after all.
What we came up with
As we were tackling this problem from 2 sides we came up with 2 similar approaches, but then leveraged the benefits of each platform to achieve our goals.
For Source web, we would add a view into the Source Web Doc Site when viewing a particular component that let you see whether a component was being used by Contentful, and what version was being used.
For Contentful, we would do something similar. Using the app framework, we would create a sidebar app to appear next to a content entry to look at what content model it was, and then work out what components were being used and then show the version info.
How did we do it
To do both of these approaches we would need to work out how to share information between our 2 platforms. Luckily, both our platforms publish packages that we could use to gather information so it shouldn’t be too hard. Or so we thought. See challenges below 😉.
While the implementation of both sides of this solution were different, they have some common ways to source the data to be used.
I will detail some of the changes in approach we had to make in the Challenges section below, but what follows is what we ended up with as our ultimate solution, after much problem solving and trial and error.
Gathering the data
Getting the data we would need to achieve our goal was probably the most challenging part of the process.
We had previously attempted something similar in our day to day Contentful team but struggled with finding an efficient way to get the data from Source Web as we didn’t necessarily know the relevance of all the data that was available. This was where the power of the hackathon comes in and the benefits of how our team was structured with expertise from both areas on hand.
Let’s get into the detail.
If we start with the mapping from Contentful data, in the majority of cases, the names of our content models can be matched up with the component it uses.
For example, we have a content model called standardBanner which maps to a component in Source Web with the package name of @vfuk/core-standard-banner and the component name of StandardBanner. So with a bit of regex and some find and replace, we could link the model to the component. In most cases anyway. More on that later.
Both platforms, Source Web and Contentful, publish packages to our internal NPM registry, so these packages are available to download and interrogate.
We used the cross-spawn package to enable us to run commands like the npm view command to gather information from the packages directly and request information on specific versions — not just the latest.
For instance, each Source Web component has its package.json file containing information we could use.
Included here is version number of course, but also the link to the docsite that contains this version, the path to the component in the repository which might be useful, the categorisation of the component (e.g. using atomic design principles, organism, molecule, atom etc.) which could also be useful.
Similarly, in the Contentful mapping package.json, we list all the dependencies and the version we are using (usually an older version than the latest — for now).
So with all that data available, and with a bit (a lot) of data processing and parsing of json data we were able to turn all this information into a pretty detailed summary of the link between Contentful and Source Web.
Now we could work on our respective front ends to create something that our users could see and use.
The end result
As the front ends for our 2 solutions are using React for the most part, I won’t go into too much detail other than where there is some interesting information to share or some challenge we needed to solve.
With that said, let’s start with Source Web.
The Source Web docsite is completely within our control so is very customisable meaning we were able to take the data we had processed and use it in various ways. We can create bespoke docsite specific React components to display any useful information to our users.
The initial view that was created is a full page view of all the components in Source Web with a column listing the Contentful Version.
You can check the status, i.e. how out of date a component is, by looking at the colour of the version badge. All of the ones on the following screenshot are red because they are very out of date. We do have an example of a positive badge though which will show as green, so stay tuned for that shortly.
In addition to this, when looking at an individual component in Source Web, we added a way to see the Contentful status to the already existing “More info” section that appears on all components.
Here is the Fifty Fifty Banner with more details about the version Contentful is on with a link that will take you to the relevant version of the docsite with a snapshot of the component at the Contentful mapped version — so you can see the props and other settings that were available, or missing, at the time.
And here is the mythical green badge that I spoke of. All good in terms of versions for StandardBanner.
And finally, we have the Contentful based solution.
As discussed previously we are using the Contentful App Framework to help us achieve our prototype on Contentful.
We already have an existing custom app live so we used that as a template to get us started, and adapted that to suit our needs. We are using React to render our apps and we are also taking advantage of the Contentful created design system — Forma 36 — to allow us to create an app that fits seamlessly into the existing Contentful UI. After that it’s just a question of making it look good and work how we want.
The main hurdle was how to get the data we need and for it to be as up to date as possible.
We started by trying to run the scripts required to gather the data within our React components. In hindsight, we probably should have realised this wouldn’t work as we’d hoped as we obviously needed to be able to run the app in the browser.
We quickly pivoted to having the creation of this data moved to a pre-build step that would be run before webpack did it’s bundling. And even better, this step would also be called when we were deploying the app on our CICD pipelines so would be kept up to date.
The Contentful app framework has a few locations where you can put apps. As we wanted the app available on any content entry and to be easily visible to the user, we opted for an app in the sidebar.
And here is our new app showing the details of the StandardBanner content model and how the versions compare to the latest Source Web. Clicking on the links in the badges with the versions in, takes you to Source Web docsite in a new window and takes you straight to those versions so you can compare the props and any other details.
And of course, when there is an out of date component, we flag that in red similar to the Source Web docsite view.
As we can only show the details where we have been able to work out the components in use, and because we have certain content models that don’t have any direct link to Source Web, when we can’t find a match in our data, we display that to the user.
And finally, because we really liked the full page view we created in Source Web docsite, a last minute addition to the hackathon was that we added a full page app in Contentful too.
The handy thing about Contentful app locations is that we can have 1 single app, but choose to display it in different locations, simply by handling this in the root of our app.
We have set up the app in the Contentful config to be allowed on multiple locations, and then in the app, we just detect which location we are using and render a different component. In this case the Sidebar component for the sidebar, and the PageView component for the page.
Then it’s just a question of creating our interface with the Forma 36 components and using the same set of data we have for the sidebar, just using all of it and not filtering it based on the current context of the content entry you are viewing.
In this screenshot you can see many similar features of the sidebar app, including the links through to Source Web docsite, the colour coded version badges (thank you StandardBanner for the green) and the visibility of those components that have no direct mapping.
This was a handy last minute addition to the hackathon and this alongside the sidebar version is a great complement to the similar views in Source Web docsite.
Challenges
In my explanation above I may have made this all sound pretty straightforward. But what I didn’t mention was some of the challenges we faced in getting to our working prototypes.
Old dependencies
Ironically, the differences in versions from the Contentful mapped version into the latest Source Web version caused problems with our initial attempts at coming up with a data gathering solution. The outdated versions caused problems when trying to include the mapping package as a dependency of Source Web.
Our solution — As detailed above, we moved over to using npm view to directly query the packages rather than our original plan of importing the dependencies and using something like fs to read the json file and process the data.
Client-Side Rendering
The system itself faced difficulties getting package information directly in the client side of the application.
Our Solution — Add a pre-build step to get the information we need at build time that can then be used at runtime in the client.
No direct mapping for certain content models
Certain content models do not have a one to one mapping to a component and our initial approach doesn’t allow us to see those. And in some cases, due to the nature of the components and props, we will never see certain components in our data.
Our Solution — Add a label explaining that there’s no direct mapping, and leave this for later. It will involve a more complex way of getting the data from compiled packages. For the hackathon, we can live without it, for now.
Future improvements
Composite Components
In our Contentful maps (e.g. Device card), a single content model can be built up of many different components. How do we link to the docs when it’s not a 1:1 map?
International Compatibility in Source Web
Allow integration of the Contentful system between different business areas and international groups.
Enhancing the page view to add more options
For instance allowing to view by components first rather than the current view of content models first, and sorting by the most out of date components to help prioritise the backlog of version bumps.
Since the Hackathon
We have been able to put our Contentful app into production which we are very excited about and is already showing benefits.
We were even able to add in some of the enhancements we wanted to make like handling any indirect mapping by adjusting our approach to getting the data from our mapping package.
We’ve demoed it to the content editing team who had positive feedback, and a few enhancements they would like to see.
The Source Web version is coming very soon too.
Conclusion
Overall, my (and my team’s) experience of Hack to the Future was extremely positive. It was great to be able to work alongside colleagues we typically don’t work with and be in the same room together after a long time apart.
It’s really satisfying to have been able to create something that has already been put into production and is proving useful and it’s unlikely we would have been able to do this without the hackathon to enable it — at least not until we found some spare time in the team (which is hard to come by).
I look forward to the opportunity to get involved with the next one.
