Ruby CLI Scraper
I have FINALLY reached the end of my first project week. The goal was to “build a Ruby Gem that provides a [CLI] to an external data source.” Easy Peasy, I just did this in my recent assignments. That was the thought, until I tried to set up my local environment. Which also wasn’t horrible. It only took 2 hours and I followed a very helpful walkthrough found here.
Okay I was through that, phew! But then came time to use the local environment to start coding. I could not figure this out! I was on a mission to get my project started ASAP, because I knew it would take me a while to execute the necessary steps (when I get stuck, I’m stuck… for a while). So I made the smart choice of forgetting about the local environment and just using the in browser IDE on Leran.co. I followed this video to set that up, and voila I was in business. That was honestly the hardest part. However, it did take me a while to come up with an idea for a project. I could not think of any websites I used regularly, nonetheless ones that had details on it that a user would want to access. I decided to scrape the Extraordinary Finds section on Amazon.com. I was super excited, until it was brought to my attention the potential of being marked a hacker for scraping Amazon. I was somewhat reassured it could be done, but I didn’t want to risk it. So I stumbled across a webpage I NEVER look at. Not as exciting, but I knew I could get the thing done using Huffpost.com.
I scraped the Top Stories section of Huffpost.com. A user can request a list of articles in Top Stories, select one based on its title, and read the whole article from the command line of the terminal. Still pretty cool in my opinion, as someone who doesn’t read the news. You can see how it functions here. Enjoy!
There were several snags along the way. Many times I was stuck but these are the things I learned:
1. Your community/network is important.
I somehow got lucky to find a great group of peers in my cohort that were there for me every step of the way. Find those people and stick with them! Together you will get through it.
2. If you have an instructor, in my case my cohort lead, utilize the help they are offering.
I am not someone who asks for help easily. So reaching out to my peers and cohort lead was not my first inclination. But the times I did, I’m grateful for because there was no time to be struggling alone.
3. Plans change, you may have to restart your project at some point, and that is okay.
I had a several hurdles thrown my way. I had to accept that things change and I can adapt. Adaptability is a great skill to have.
4. Google is your BEST FRIEND.
Go Best Friend, that’s my best friend! Learn how to ask questions in the right way on Google. It is also a great way to learn new things and see how other people solved similar problems to yours.
5. Lastly, believe in yourself.
This has been my mantra since I began this program. I went into this project feeling completely clueless, with HIGH anxiety. But I got to the other side.
In the end, I created a project that I am proud of. It functions the way it’s supposed to, and I have a better understanding of object oriented programming, as well as belongs to and has many relationships, which was the purpose.
