Introducing Headlyne.me

Or, How a Marketing Guy Taught Himself to Code the Hard Way

When I was 12 years old, I picked up a book– a really old, dusty copy of “Learn Javascript, 12th Edition”. Having extensively used the Internet but never having had an opportunity to learn the actual code, writing scripts for radio buttons and polls in the windows text editor was like a child successfully riding a bike for the first time without falling down– it was eye-opening and tremendously inspiring.

But unfortunately it stopped short there. I never did get past chapter 4, and that was that.

Fast-forward 13 years later– I’m a marketing guy who hopelessly loves watching the latest web technologies grow, and Hacker News quickly became a haven for my passion for web (and my way of coping with the fact that I, in fact, could not code to save my life).

My roommate called me over to show what he was working on for his hackathon– a sentence-parser that would intelligently break out sentences into individual, grammatical components to use with a search engine. It was built on an engine I’d never heard of before, called D3.js.

An example of sentence parsing: breaking down words into recognizable grammatical “items”.

I felt like a kid again– imagine what you could do with this kind of technology! And there I was whisked away back to my 12 year-old self. I decided that I was going to make my first decent project, an app to visualize the top news going around the world. The motivation for this came from the fact that the majority of people I spoke to had no awareness of world affairs, never mind any interest in them. I figured this would be a good opportunity to expose them to news in a more interactive way.

I scribbled this naive statement to myself on a post-it: “Learn Python to build visual globe for news,” then got right down to studying Python fundamentals on Udacity. Alhough useful for learning syntax in theory, not having a project made it difficult for me to apply this knowledge so I thought I’d throw myself against the wall to try and try to bootstrap the project with the little knowledge I had.

Boy, was I naive.

There were multiple problems:

Me about 7 weeks ago

• I had to build out some sort of way to extract top news, and their titles and links. And what source of news was I going to use?
• I also had to figure out a way to parse articles to figure out where I would place the marker for each respective article.
• Where would I get coordinates to display the articles?
• How the heck was I going to build out a globe anyway? Did Google Earth have a solution?

At this point I was ready to throw up my hands in the air, but my roommate came to the rescue and suggested AlchemyAPI for parsing through the list of articles to obtain the most relevant nouns. This was the key. After annoying their customer service representative numerous times, and with the help of Learning Python the Hard Way, I finally got it to detect the countries from each of the article’s titles from reddit’s r/worldnews forum, which I figured would be the best because it has a quasi-democratic method for determining the most interesting articles.

But this approach had a key flaw– what if the title didn’t have a country name or city name in it? Bugger.

Discouraged once again, I turned to a software engineering buddy from Facebook and he had a brilliant insight– why not just run the API through each article? Yeah, why didn’t I think of that? AlchemyAPI can be instructed to obtain the most relevant noun, so I went ahead and re-wrote the Python script to search for the most relevant nouns classified as a Country or City.

AlchemyAPI’s a pretty cool tool, actually.

Next problem: Coordinates. This was thankfully more or less solved by Google Geocoding. But again, without fail, FATAL ERROR, it announced.

After hours of grappling with their documentation, I noticed in the logs that it always cut off after exactly 20 articles (Reddit shows 25 by default). Then I stumbled upon something called usage limits, which actually has a cap on a per-second basis. I suspected this had to do something with me overusing the API, so I learned how to force the program to delay the execution through time.time()– and like black magic, it worked. Somehow.

So now I learned to scrape the data I need– article titles, URLs, country names, and their respective coordinates. Time to make me a globe.

Turns out, it’s tough business making a globe. Google Earth is the most obvious tool that came to mind, but turns out they can only display the intensity of data (as opposed to metadata) without an excessive amount of tinkering.

The D3.JS example I was suggested. Turns out Javascript is a bit of an arse.

My roomie suggested that I try D3.js, as it had an innately configured Earth module combined with Topojson. But after about a week of trying to understand the javascript code, I gave up– there was no way I was going to understand this, even with the help of codeacademy.

I coincidentally ended up on a search result at work the next day when I was reading an article about how Safari would be supporting WebGL OOTB starting with Yosemite, and found the answer to my prayers: WebGL Earth.

And the angels descended from the heavens and said: “Thou too, shall have WebGL”

As unelegant as the website and documentation looked, it had exactly what I needed– marker support, 3D tile overlays, and even zoom controls out-of-box. Moreover, the marker text used plain HTML to display the information, which would make it easy for me to draw text from Python to display.

So… about that. How exactly was I supposed to do that? Plugging in Python into an HTML file?

StackOverflow suggested Django, which was supposed to be the Python equivalent of Rails for Ruby. But someone on Reddit at r/learnpython suggested I go with Jinja.

Turns out this was just the code to literally plug into Python. I still needed a microframework to connect the HTML file to Python and have it acknowledge it existed.

Not another language, goddammit.

Turns out the answer was Flask, which thankfully wasn’t anywhere as difficult as Javascript (and, as I later found out, technically wasn’t a language). With a built-in test server, and with the help of my friend from earlier, we corrected the code for silly mistakes (e.g. plugging in an entire list into an array as one list item, silly me).

Then, it came to life.

My baby. All this work had finally paid off. The blue marble sitting there rotated serenely as I couldn’t believe my eyes.

With the help of github documentation, I quickly got the app live on Heroku, where it resides today as a beta. You can check out the full code on github here.

I eat fatal errors like you for breakfast

This has been an immensely rewarding experience in and of itself. Sure, there’s that nice spinning globe, but still needs tweaking (like why does the server crash sometimes? or, why do some of the Earth tiles go missing? or, how can I make this code more efficient in less lines?)

I’ve learned some key lessons that will stick with me:

1. Programmers aren’t better because they “know more languages,” they’re better at those languages because they’ve used it longer than you and actually know what doesn’t work. They know the couple few methods that end up working.
2. Programmers are better than plebians because they have best practices. Like not looping through an API for requests multiple times when they can do it in one. or writing individual functions for multiple purposes rather than one gargantuan function.
3. The secret to learning good code isn’t how many classes you take, or for that matter how many lessons you get through– it’s actually creating an app, and in the process learning how to use the damn error log.
4. One of the most frustrating things about a non-engineering person trying to break into coding was that not many people were willing or able to explain how to accomplish something I was trying to do. I could’ve heard “Oh, you need to first write a program using Python to scrape titles, urls, and entities using AlchemyAPI, then run it through a coordinate dictionary like Google Geocoding, then pass those values to an HTML file using a templating language, then embody the app via a microframework like Flask, then upload it to a Virtual Private Server because Python doesn’t work by itself, it needs to be executed!” or something of the sort. Sure, I might not have understood all of it, but providing a sense of scope and direction for how I would complete the project would have been useful, rather than getting instructions 1 step at at time. Then again, I’ve learned now that this is how hackers/web devs work– find solutions to 1 problem, 1 step at a time, rather than try to plan out an entire project in advance.

I don’t know how you guys pull it off, web devs. But my appreciation for your work has escalated exponentially.

Phil out.