Parsing Data from Socials: “Interactor” Gem
A few years ago we had an app which was about parsing data from social media. The working principle was simple. A user linked his social media profiles to the app. It received some data and displayed it in a one single feed. The challenging part was to come up with a solution that would parse user data and save it in one format. But because each social media has its own format of data we had to somehow normalize it first. For example, someone’s feed might match our table and someone’s might not.
As a solution we decided to implement the abstract factory pattern, because it engages the capabilities of inheritance and overriding to the full extent.
This is how it worked. A basic worker received a profile with an access token to a social media, and an endpoint (this is what the worker had to parse, i.e. a profile, a feed or friends). We divided the feed into 2 parts: personal feed and home feed (where you could see what your friends post).
Depending on the endpoint, the worker selected a grabber for a particular social media. The grabber made extra requests and, if necessary, managed search parameters and normalized data (some gems return results of their objects). Received, the data is put into a module that processes raw data received from the grabber. You can see it below:
Such structure was quite simple and understandable. But eventually we had to change it because of social media specifics. In addition to that, the way of processing this data had to be changed as well. We needed not only to convert it into a common format but also make user filters process it. Long story short, the code grew bigger and became less logical and readable.
Initially, the aim was to keep to the DRY (don’t repeat yourself) principle. Which we did. Almost. The number of functions grew bigger and bigger. Eventually, the whole thing became quite a mess.
Thus, we faced a very important principle — single responsibility. It means that theoretically you should design a class in such a way that changes would be necessary only if it stopped working. The class should also have high cohesion.
Solutions?
First, we need to understand what this gem does. It allows you to describe services while you solve some problems from the inside, and the gem solves all the other ones, e.g. how to link the services, how to transfer data between them etc. So you don’t have to work on all the problems at the same time — you can work on one issue in each service.
Let’s take a closer look at how this works. Here is a test sample that receives a feed from twitter and outputs some of its fields: “id”, “body”, “photo”, “links”. The “Interactor” gem allows for describing the task as a sequence of simple steps.
Say, we have some token — “secret”. We hardcoded it to make things simpler. We also have the “Interactor” (this time it’s “ReceiveTwitterfeed”). We need to call it and put auth-token and auth-secret in it. Succeeded, we assign some variable from the data to the instance. In the case of failure, we receive “fail”. So there is just rather tiny and simple controller.
Here we can see that the “Interactor::Organizer” is connected, and after that the “organize” method is called. The “Interactor” gem provides 2 types of connected modules:
- Organizer;
- Interactor.
The organizer is used to organize the steps, i.e. the interaction chain of these interactors. If we want to receive a twitter feed, we need to grab it from the social media and process it. There is hardly anything challenging about it. The “Interactor” gem is responsible for the auth-token and auth-secret to be displayed everywhere. That is both of these parameters will be available in each interactor.
Let’s have a look at “GrabTwitterFeed”.
What we see here is just the Interactor. It is necessary to define the call method in it. There are no input parameters — all of them are available through the context object. Now we begin the execution. We’ve got the “api_client” and we want to get the “user_timeline” method result. The API Client is just a twitter gem. Then the user_timeline puts it into the “twitter_feed variable”. And if something goes wrong, it will tell us that all failed and this chain won’t work.
At this point, we got raw twitter feed data. Next, there was “ProcessTwitterFeed”.
As you can see this class is quite simple.
“ProcessTweet” — is another organizer that describes how to process a tweet.
“ProcessTweet” describes the way a tweet should be processed. That is whether we should normalize data, save the photo, save the shared links or save the tweet itself.
As you see, each of the steps is quite simple, and even if there is a bit more real logic, it would hardly affect any of the services.
Moreover, the “Interactor” gem provides a small but very useful DSL. It allows you to add actions to be executed before or after the basic logic starts. For instance, we needed to put function calls all over the code for profiling, meaning we searched for a bottleneck, so they remained. The “Interactor” gem allows you to do it in a faster way:
Conclusion
Basically, the “Interactor” gem is designed to solve some quite important problems, but the very implementation leaves much to be desired.
First and foremost all the services, written with this gem have high coupling. They all have the same context through which data is transferred. Meaning the context data is available in each service. The gem also “knows” that this data will be added to the context, so it partially “knows” about other implementation.
Second, class organization. It is not the most convenient way to name a class basing on the problem it is to solve. It would be much more readable if it were a symbol which would later be changed to classes somewhere inside.
We can also mention that there is no way of knowing whether it is an organizer or an interactor. Plus, namespaces are added on the interactor’s side, as it is not quite convenient to put them in a folder.
Moving on to the next point: organizers can’t assign control logic. The chain element has to determine whether it is supposed to be called at all within itself.
If something goes wrong the “Interactor” gem has its own way of processing in store. It calls the fail method, wherein you can also describe the rollback method which works in the case of failure. It is hardly convenient comparing to one’s own exceptions. Such places in the code where something might happen are clear, and custom exceptions describe them quite well. But this aspect is lost in the “Interactor” gem.
