Development with people without an IT background
Applying IT skills to karaoke parties
A cooperation story between people with and without IT background, to ease up the preparation of karaoke parties
Coming from an IT background, we tend to look into things around us and try to jump in on the next big thing: the newest library, programming language, framework, and so on. However, sometimes we can look into smaller things/events around us, and with a little amount of effort on our part, we can have an impact in easing up other people's lives, especially when those people do not come from the same background.
The situation
Due to new life circumstances, I started taking part in a new circle of friends who quite often like to organize karaoke parties. Since this group does not come from an IT background, I quickly figured out that some points could be improved in the preparations. For example:
- The catalogs were being maintained fully manually with word documents and passed to the event participants as PDFs. This meant that the only way to search through the catalogs was via the find option in the PDF reader
- Every time a new song was added a number needed to be assigned. This also meant that they needed to keep track of all deleted songs in history to know which numbers were available. This was also achieved via a word document filled manually
- As the numbers were all assigned manually, there was a fairly high possibility that due to human error multiple songs would get assigned the same number
Phase 1 — Proof of Concept (POC)
For the first concept of the app, the following steps were necessary:
- Load the catalogs as a word document in some data storage
- Return those catalogs into a UI that would allow people to filter and sort the content easily
- Use some free hosting and data storage to deploy both the app and the frontend
Catalog loading
Since the structure of data stored wasn't a problem and the data was not hierarchical, a caching solution (Redis) was used. This would be a decision that would come back to haunt me down the line, but it did fit for POC.
Allow searching/sorting via UI
There are many libraries out there that provide features like sorting, searching, and management of data in a table, so implementing this kind of functionality on our own was never really seen as an option. With this in mind, the library ag-grid (https://www.ag-grid.com/) was used. This library provided out of the box filtering and sorting for the data that we add in there, and it has quite extensive documentation and feature set which could help cover future needs
Hosting and data storage
Since the code was stored in GitHub already, hosting the frontend in GitHub pages seemed like an easy and convenient option. Hosting the backend had a couple more alternatives, but from the moment we were able to use Docker containers, any free service would suffice. In the end, Heroku was chosen, as not only it is free and requires little setup, but also it already has integrated solutions for data storage (for example a Redis solution for caching), which made the setup easier
Feedback for phase 1
Now the POC was out and ready for the initial feedback. The response was very positive all around! They loved what they could do already and it was seen as an improvement over the current situation. But of course, there were points for improvement, namely:
- The information about song categories from the catalogs was lost on the app. This had to be recovered
- By loading the catalogs all together it became evident the presence of duplicates or overlapping numbers
Category management
As the information of categories was already provided in the original word documents, this was just a matter of making sure this data is fetched and changing the UI to also show it and filter based on it.
Catalog management — duplicate checks
Managing the catalog required a lot more insights into the data, which we could not get simply from raw cached info. This triggered phase 2 of the project.
Phase 2 — Generating insights and catalog management
We needed to have some structure in our data storage to generate insights. Storing all information raw and then processing it in memory would just not be scalable if we wanted to address catalog problems like duplicate entries as well as checking the available numbers after a song was deleted. With this in mind, and since we could still benefit from a hierarchical structure, we decided to go for MongoDB. Since MongoDB provides a free tier and it's also very easy to connect locally by taking advantage of Docker containers, it also simplified the future development.
So now the data was not in cache anymore but in a MongoDB collection, how can we extract those insights? Here are some of the pieces of information we wanted to know:
- Are there two songs with the same number? Which ones?
- Do we have songs that appear in multiple catalog files? Can we merge them?
- Are there gaps in the number sequence that we have for our songs? If yes, which numbers are available?
Songs with the same number
By changing the data to use an actual database where we could query on its attributes like the number, finding songs with the same number became an easy problem to solve, as we could just query for all the songs where the same number appeared more than once. Once we have that, we can produce a list of all those possible duplicates. This list would then be shared with the original author of the catalogs, who could then analyze whether this was a mistake and they needed different numbers, whether the song appeared as a duplicate by oversight or if it was on purpose.
Having this check proved to be very useful as future additions to the catalog could be run against it multiple times, giving us immediate feedback
The same song appears in multiple catalog files
This was a real scenario was we had two catalog files, one for songs in the French language and another for English. A song could appear in both catalogs provided they had parts of it in French and another in English for example.
After communication with the owner of the catalog, we agreed that a song with the same name, number, and artist appearing in multiple catalogs could just be merged into a single entry, which would have a reference for both catalogs
Gaps in number sequence
Since we now have the list of all assigned numbers together, we could query our list to find which numbers were not assigned and report a list with these numbers. As a result, the document where the deleted songs were being kept for future sake was made obsolete, since we could now obtain the same list in less than seconds, with no manual error
Conclusion
In this article, I went through the process of looking at a problem, establishing a POC, gathering the initial feedback, and iterating the POC into a more stable and useful version.
The app itself can be found here and the code can be found here
This is the first article I have ever written for a tech platform, so any comments or questions and sharing would be more than appreciated!
