World’s Top Movies: My first CLI Ruby application project

Photo by Jon Tyson on Unsplash

I’ll share my reflections on the Ruby learning process of my Software Development studies, including my final project called World’s Top Movies.

What did I learn in this period?

After completing my front-end Pokémon Matching Cards project using JavaScript and its ReactJS library, it was time to dig into the back-end. Ruby was the designated open source programming language to accomplish this mission. The reason? Its simplicity and productivity, accompanied by a natural-to-read syntax.

The adventure started with all the usual topics that most programming languages have in common: Variables and Methods, Booleans, Logic and Conditionals, Looping, Arrays, Iteration, Hashes and Data Structures. Here I could see with my own eyes the magic of already having experience with one programming language (JavaScript) as I managed to review all of these topics in just a few days. It was more a matter of adapting to the new syntax than the logic behind it.

After completing the procedural Ruby concepts, I dug into the topics in Object Orientation, where I learned about Class Objects, Object Attributes, Self, Class Variables and Methods, Object Relationships and Inheritance. I also learned some of these topics during my JavaScript learning, but I could see why OOP can be so powerful this time. As a bonus, I learned the basics of Regular Expressions.

From that point, things started to be more challenging in my point of view. The following topics to review were Metaprogramming, Configuring Applications, SQL, Object-Relational Mapping, Using ActiveRecord, Getting Data from Remote Sources and Rack. It was heaps of information to digest, but I did tons of coding exercises to grasp these better.

My Project: a CLI Application called World’s Top Movies

The initial requirements of the project I did were the following:

1. It had to provide a Command Line Interface.
2. The application must provide access to data from a web page or an API
3. The data provided must go at least one level deep.
4. Use good Object-Oriented design patterns such as creating collections of objects, not hashes and avoiding scraping data more than once per webpage to improve the application performance.
5. Optionally create a gem from the application.

I wanted to go the extra mile, so I added database functionality, and I used SQLite3 to accomplish this.

Structure and Features

My project is a CLI application that utilises Web Scraping to obtain data from IMDB, an amazon online database for television, video games, and especially movies. The application scrapes top-rated movies worldwide based on IMDB user reviews and displays them on the CLI, and also has a database that helps users store their favourite movies and leave notes.

Database schema

As I said above, one of the topics I learned during this learning period was Active Record, which is an Object Relational Mapping framework. I used this Ruby gem because it allows me to handle relational databases quickly and efficiently. It’s a link between Ruby and databases.

The three main tables I used were:

• Users: stores the usernames
• Movies: stores all the favourite movies and movies with notes
• Notes: stores all the notes messages given by the users and the movie associated with each note through the movie_id foreign key.

In order to create the associations, I had to create two tables.

• User_movies: links users with movies by taking the user_id and the movie_id foreign keys
• User_notes: links users with notes by taking the user_id and the note_id foreign keys

Database schema

The application’s structure consists of the following sections:

• Introduction & sign up or log in
• Top Movies Lookup
• My Favourite Movies
• My Movie Notes

Now that you might have an idea of the models, I’ll explain each section from the user’s perspective.

Introduction & sign up or login section

First, the user will see the title, my contact info, a short introduction, and immediately after that, they’re asked to log in or sign up. The user types a username, and the application checks whether that username is stored in the database. If stored, it logs in; otherwise, it creates a new username and saves it in the database. After this action, the user is asked whether they want to go to the Top Movies Lookup, My Favourite Movies Section or My Movie Notes.

Introduction & sign up or log in.

Top Movies Lookup section

If users go to the Top Movies Lookup, they’re asked to see the list of top movies by genre or a general lookup (not considering genre but merely by rating). Then the application does web scraping to retrieve the world’s top movies as per user specifications. To scrape the data, I utilised the HTTParty gem to get the response from a GET request and Nokogiri (鋸) for parsing the body of HTML responses and work efficiently with XML.

A user chooses Top Movies Lookup to look up movies by genre.

Then the user is asked whether they want to see the detailed info of any movies from that list or add any of them to their favourite list. Additionally, they can leave and start a new lookup, go to My Favourite Movies or My Movie Notes sections, or print all the movies that have been displayed so far in case the user has looked up more movies.

If the user opts to see the details of a movie, they’ll see what’s displayed in the following screenshot, and finally, they’re asked if they want to add that movie to their favourites or leave a note. If a user adds movies to their favourites, the application will first check if these are already linked to that user in the database. The movie is added if it’s not linked yet; otherwise, the user receives a prompt message stating that it is already stored in their favourite movies section.

On the other hand, if users want to leave a note, they can type it in and hit enter. The note message will be automatically saved in the notes table and linked with their user id in the database.

A user chooses to see detailed info about one of the movies and leave a note.

As said before, users can add multiple movies to their favourites. So, they don’t need to see the detailed information to do so.

A user can add multiple movies to their favourites.

My Favourite Movies section

This section contains everything to manage the user’s favourite movies. They can print the list of their favourite movies, see the detailed info about any of them or remove films from their favourites if they wish.

If the user wants to see their favourite movies, the CLI will display the same view as looking up movies in the lookup section. I tried to be “DRY” whenever I could, so I used the same class method from the Movie to do both actions.

The list of a user’s favourite movies

Similarly, if the user wants to see detailed info about one of their favourite movies, the view will be the same as the movie from the lookup section. The only difference is the bottom part, as they can see their notes (if there is any). It’s worth clarifying that the movie doesn’t have to be in their favourites to leave notes. I’ll explain more of this in the My Movie Notes section.

Detailed information of a favourite movie which also has a given note

As said before, the users can also remove movies from their favourite list. This action is done by deleting the association of the movie id and the user id from the database.

Users can select the movies they want to delete

My Movie Notes section

In this section, a user can see all their notes and the movies associated with those notes. They can also see detailed info on any of the movies with notes. Last but not least, they can delete any of their notes.

If users want to see all their notes, they’ll see them ordered chronologically with the associated movie and the date and local time the note message was left.

Notes list from a given user

As said above, users can open a specific movie and see detailed information. Since these movies have note messages, they can see the notes associated with their user ids at the bottom. And if they want to delete any note message, they can also do so.

Users can the notes of a given movie and delete any notes from any movie.

Styling

You probably might be asking what about the colours, font styling, CLI prompts and title style. For the colours and font styling, I used a gem called Colorize that does an excellent job. For the CLI prompts, I used the TTY-prompt gem. And for the title, I used a gem called Artii.

Challenges

During the development of this CLI application, there were some adversities that I had to overcome:

1. Finding the right CSS selectors: the process of finding the correct CSS selectors was a significant hurdle. There were cases that CSS selectors worked from one webpage but didn’t work for another one. For example, some movies have official websites, and others don’t. This situation made me find ways for using CSS selectors that wouldn’t affect obtaining the desired information.
2. Bugs for all possible scenarios: During the development, I had to constantly think about what could go wrong depending on specific scenarios. One worth mentioning is when two users have a movie in common, and one of them decides to remove that movie from their favourites. I had to make sure I deleted movie records from the user_movies table and not from the movies table. This case seems obvious, but I wasn’t considering this and the movies were actually deleted from all users. Another scenario was that users shouldn’t access the My Favourite Movies and My Movie notes when they don’t have any favourite movies or notes. Using pry was the ace under the sleeve. I learned a lot about how to debug, thanks to this tool.
3. String manipulation: For some movies, the website didn’t display some of the data the way I wanted as I needed them stored in my class instances and database. For example, numerical values such as the number of votes had commas when I needed to store only the numbers in the movie instances and records. Another case was that commas separated the genres and stars, and I wanted to store them as arrays, and the year was inside parentheses when I only needed the numbers. I used some basic regular expressions and ruby methods to retrieve the data the way I wanted.
4. Defining appropriate relationships among models: When defining how tables were related, I had to review some relational database concepts such as belongs_to, has_many, has_many_through, has_one and has_one_through, as well as dependent records from tables. For example, if a note is destroyed, then the user_note record associated with that note must be destroyed as well.
5. Creating a gem for this project and publishing it on rubygems.org: This task was very challenging. I had no idea about how to create a gem, so I had to read the documentation. I ended up using Bundler to create the gem. This process was far from being straightforward, perhaps, due to my lack of experience making gems. One of the issues was that I had to remove the require_all gem as it seemed to work only for development, so I had to redesign all the require and require_relative references. Another one, and probably the hardest to figure out, was how to connect the database to the local folder of the user who installed the gem. Using __dir__ was the key to solve this issue (thanks on this one, Sha!).

To sum up, I encountered myself well pleased with my results during my Ruby learning process. The hours I faced during the learning period and especially during the development of the CLI application make me feel confident with the skills I have gained. I’m ready to start my journey of learning Ruby on Rails!

Here is the gem repository: https://github.com/CodeHunt101/world-top-movies

If you’d like to try and test my gem, you can find it on the following link: https://rubygems.org/gems/world_top_movies.

If you’d like to see how I built my project from scratch, open up the repository of my project: https://github.com/CodeHunt101/top-movies-web-scraping-project-phase-3