CS373 Fall 2020: Week 7

What Did I Do This Week?

This week, I continued working away at Project 2. My job is to collect data from both RESTFUL and non-RESTFUL sources, involving the OpenSecrets API, some spreadsheets, a couple of HTML pages, and PDFs.

Scraping the spreadsheets was pretty trivial — the Python module, Pandas, made things incredibly easy. HTML was a bit tougher, but PyQuery and some JQuery refresher got me through. The PDFs weren’t as hard as I thought, but they involved some manual work. 181 PDFs have to go through a program called Tabula where they’re converted into a scrapable CSV.

I’ve created a nice little workflow for myself after banging my head against the wall for a bit, and I’m happy to say that I’m further along than I thought!

What’s In My Way

Unfortunately, time seems to be in my way both in SWE and out of SWE. The 3-piece pop-indie band, Jukebox the Ghost, has a perfect song describing this.

What Will I Do Next Week?

Thanks to my amazing team, we’ve setup the boilerplate for connecting to our Postgres database with our Flask app. After data collection is finished, the next step is to migrate our data to the database.

Thoughts on the Open-Closed Principle?

The read was interesting. Object Oriented Programming seems to be a polarizing practice among online nerds, but the paper is a reminder of the benefits and the best-case usage of OOP.

Experience with Iterators, Generators, and Yield?

Like most weeks, I’m going to say that my experience with the above was interesting. My primary language has become Javascript, due to the amount of React/Node.js code I write — thus, my go-to way of modifying elements in an iterable object is very functional, involving lambdas, reducers, and mappers. The lectures about iterators, generators, yield, and list comprehension helped me get more into a more python mindset of coding.

What Made Me Happy This Week?

I am a fan of professional League of Legends, as dorky as that is, and I’ve been addicted to keeping up with the ongoing World tournament. While I usually sleep in, I’ve been waking up surprisingly early to catch the live games being played in Shanghai. With group stage games going on, I’m been a bit happier because they’ve given me something to look forward to in such a monotonous and stressful time. Unfortunately, my favorite team, TSM, is completely choking in their group which has made me a lot less happy :(.

Pick-of-the-week/Tip-of-the-week?

PyLance is a VSCode python linter that has been a lot more reliable than pylint for me. Where pylint gives me unusual errors and annoying warnings, PyLance has managed to avoid all of that so far. It needs some work, but the extension has made coding python in VSCode a lot more enjoyable.

Obligatory headshot