Tracking the weather in major Moroccan cities with Go

The Go programming language is a breath of fresh air. The things it brings to the table will amaze any developer who spent a good amount of time in C. While it is certainly not as fast (we will leave that task for Rust), it can do wonders in almost every other regard. Its concurrency paradigms are intuitive and very powerful, and the way it handles structs with receivers and mutations add a level of abstraction unprecedented in many other procedural languages, and the list goes on and on.

I just recently started learning this wonderful language and I wanted to realize a mini-project with it and share the steps I went through with anyone interested. I also tried to touch on something that concerns my country to add a layer of realism to the whole thing. Now, buckle up and let’s start!

1st task: Getting a list of major Moroccan cities and their coordinates

The first thing we need to do is retrieve a large sum of cities to work with. Yeah, sure we could manually write each one, but that shouldn’t be the first thing to cross your mind. We need to delegate as many tasks to our program and only take care of the underlying logic.

Not only that but automating this process will make this code reusable for any other country, or if you’re feeling spicy, for every city internationally.

The hard way (for educational purposes)

The first way would be scrapping the data from the internet manually if we struggle to find a reliable source. The best library to do this is goquery in my book. We can easily import it with this command:

go get github.com/PuerkitoBio/goquery

With this, we can now send a GET request to any website, and for our case this Wikipedia article, and then we will retrieve the Html elements that we need.

Disclaimer: this is done only for educational purposes, and thus is under fair use law. I’m not advocating for this to be done in any malicious or disruptive manner.

The first thing we need to do is import the packages we will need:

import(
  "fmt"
  "net/http"
  "github.com/PuerkitoBio/goquery"
)

We will be using fmt to display the results in our terminal, net/http to send HTTP requests, and goquery to parse the HTML document.

Another thing I like to do in Go is to create a function that handles errors because it can get very messy very quickly if we don’t.

With this we can write our main function like this:

The first line fetches the HTML document from the website and then we check if there was an error with the function we previously created. We then defer ending the connection, (defer means it will execute at the end of the main) and parse the document with our goquery library.

The tricky part here is the find method because a good query will let us get exactly the data we need. A general tip, in this case, is to press CTRL+U on the website you will scrape and read the source code. In this case, I already tested this one and the result will look something like this on our terminal:

the easy way

Another method which will be the one we’re going to use is a package called gountries. Instead of fetching every city on our own, (and then fetching its coordinates on our own) we can use a library made specifically for this use case.

We can get the package with this command:

go get github.com/pariz/gountries

the main function will then be as simple as this:

The output will look like this:

2nd task: Calculate the weather in each city for a specific day

The next step will be generating the weather for each city using its coordinates. For this, we can use an API because we’re obviously not meteorologists. A good one would be at https://openweathermap.org/api

We will work however with an abstraction on top of this API, and a good package to do it is this one:

go get github.com/briandowns/openweathermap

We can then make a function that calculates the weather based on geolocation:

Using this function, we can now easily compute the weather of any city (keep in mind that you’re only allowed 60 calls/minute because we are using the free option)

3rd task: Convert the results into an excel sheet for convenience

Now, we want to store these data inside an excel sheet to review at our own pace. The best package to do so would be this one:

go get github.com/360EntSecGroup-Skylar/excelize

We can then easily write a function that stores this data inside a file we will call weather.xlsx

Now we only need our main function:

And if we run the program, we get a file in our directory and if we open it we get this result:

Bonus: Automating the process to generate data every hour

If you have followed along, this is a little homework for you. Try to automate this process by writing a Go program that runs every one hour or so and stores the data in a new excel file and each file is named at the exact date & time it was generated. Have fun!

You can get the code of this project in this repository here.