Using trees for real
Introduction
A lot of software engineers go through a lifetime without using the data structure: trees. Trees can be a very powerful way to operate your data, and can drastically reduce computations (linear → logarithmic) needed for a problem. Our use case was not to minimize computations but just to organize data in a refined manner. I want to share a mundane software/full-stack project where I applied this.
Background
We wanted to integrate Freshdesk (a customer service software providing helpdesk support) into our platform so that our users could connect with our customer-support agents. We have custom UIs built, which interact with Freshdesk APIs to perform operations.
The backend service is called Blender, and the architecture is straightforward. Our frontend calls Blender via HTTP, Blender then calls Freshdesk APIs, there’s also a local database where we cache data. We are only storing ticket fields and not conversation/chats. This helps us serve all basic ticket information to a user without hitting the get tickets API from Freshdesk (we use Freshdesk webhooks to keep this data in sync).
One specific API which was very limiting to us was the Freshdesk attachment API: they have a limit of 15MB per attachment. We want our customers to send us much larger attachments, such as a video of their gameplay/issue. The idea is to use AWS S3. Uploading and storing files on S3 is straightforward, but how do we integrate it? What are our challenges:
- Systems (Gamezop, Freshdesk) should be aware that an upload is done on a ticket.
- An upload can be done at any point in the conversation. We also need to keep the state of when the upload was done. This information will be used by the frontend to display the upload in the conversation in the correct order.
- The uploaded URL should be accessible to the agent hassle-free.
The Solution
Do not keep track of the uploads that have been done on our databases. Instead, let’s create a private note (this is something that Freshdesk and most CRM tools provide) and parse some metadata on the private note to figure out an upload that is present. The private note approach gives us two advantages:
- We no longer need to store information about the chat. When we display conversation on the frontend, we hit the Freshdesk Conversation API anyway. Metadata stored in the private chat can be used to derive that an upload was done here, thus returning the same information to the frontend.
- Private notes are accessible to customer support agents so the agent can easily access the upload.
Initially, I just stored a JSON full of data on the note. Then, when the Freshdesk Conversation API was returned, I would try to parse a JSON on every private note and figure out information.
{"uploadKey":"/s3/path/to/file","getURL":"http://blender.skillclash.com/extras/getURL?BLENDER_S3APIKEY=...&tid=..&uploadKey=..","tid":1,"message":"please help me with this issue"}
My manager requested that we have a bold header, relaying to the agents this note shouldn’t be deleted, and the data should be a little more readable. Agents would have to click on the link and read the message sent with the attachment.
This is what the final output looks like:
I decided to write my own data interchange format like JSON/XML/HTML.
Here’s an example of what that looks like:
Having a fully custom tokenizer + parser for a data interchange format allowed me to be very structured with my information. I decided that the (meta) tag would have JSON, and the (message) part would have YAML, which makes it readable. There are also tags with (^ and ^) which denote the start and end of the data. These tags contain long hashes known to Blender, which adds an extra layer of security to the user’s data.
Parsing the private note
To operate on this data we need to construct a tree. Here are the types and function specifications which need to be written to make this work:
blenderParser uses a stack to construct a tree, returning the root node of the tree.
Conclusions
The tree implementation took me about 2hrs to code up and write test cases for. It gives us a strong framework to build features on.
