Creating a program in GO to fetch the latest news in any given topic

My SLTP Walkthrough: Part 4

Photo by Roman Kraft on Unsplash

I never programmed in Go (a programming language) before. To truly explain my thought process I will write this article assuming I have never written a single line of code in any programming language. How do I learn stuff? Can I create the program I want without doing any of those online courses or boot camps? I will try to write this tutorial in such a way that even a high school student who doesn’t know anything about programming understands how to self learn and build real stuff. With that being said, I hope even experienced programmers gain some insight here.

So, what I want to build is basically a piece of code that takes some input and gives an output. The specific use case is as follows-

Create a function to fetch the latest news in the required topics.

This is the first step in my “Generate News Posters” feature. (for context, refer to the older articles in this series listed at the bottom)

The idea here is simple. I want to make a computer program that takes some input- say a topic of interest, and it will give me back a list of the latest news articles based on that topic.

There are two levels of self-learning I want to emphasis on (something I made up myself)-

• Question-based learning (QBL)- Ask questions based on what you want to do
• Observation-based learning (OBL)- Observe what you get when you blindly do something

I will be demonstrating both these techniques as I proceed further.

The first question I have in my mind- How do I install Go in Windows?

A quick google search got me this-

Download and install

I followed the above steps and checked in Command prompt the Go version as mentioned to confirm that my installation was done. I am using go1.15.1 windows/amd64 version on a windows 10 laptop for this tutorial.

If you are even unaware of what command prompt is, just google that! You can easily access it from the search bar, typing it out.

Go is now installed, but how do I write a computer program? A quick google search will reveal that you will need a code editor to write your code. Visual Studio Code is one of the popular ones out there, which I installed.

Next, how do I write a Go program in VS Code? I found this tutorial below for the same.

GoLang Tutorial - Visual Studio Code - 2020

As per this tutorial, I installed the plugins and ran a hello world program. Now, I have official written my first program in Go. Here is where OBL comes into play. Just blindly follow the steps and observe what is the output you are getting. What should you do to get what you want?

For example, in this case, I copy-pasted the hello world program but I wasn’t sure how to actually run the program. In the tutorial, I notice that go run file_name.go will run the program in the terminal.

Realizing this is where your learning happens.

At this point, I know how to create a Go program file as well as run it. How do I write code to do what I want- To fetch news articles?

Time to ask more questions. Where do I fetch my news articles from?

I have the answer to this from my domain knowledge of digital media. I know that RSS feeds are a great source of news. So, if you are building any application as per your need and have a similar context, you will already know your answer. If not, once again browse through google about the same.

I have set up my own RSS feed from Google Alerts on the topic of Artificial Intelligence. How did I figure it out? Just by googling and exploring various things. This was again my domain knowledge. It is simple and here is my RSS feed link- https://www.google.com/alerts/feeds/11016768836951419989/3365315696613981974 (To see the XML tags, click on the link. The preview of the rendered data that the link contains is below)

If you visit the link, you will see code like text with a lot of information. It is basically an XML file. You will also notice that it contains links to the latest news on this topic. That is exactly what I want my computer program to fetch.

Before we go further, understanding the basic stuff about the code and syntax of the language is important. There are 100s of free resources for the same. I will list a couple of them here-

Go Tutorial

Go by Example

alco/gostart

You can quickly skim through them when you have some time to get the basic idea. We will still be using Google to write our code as and when required.

After a brief revision, I simply googled- “Read an RSS XML feed in Golang”.

This is the first link you will get, basically the direct solution to what we want to do:

mmcdole/gofeed

I created a new Golang file in VS Code by nametest to try and experiment with some code.

Whew! We are almost there now. I want to store some important parts of this data within the program. A quick read online will make you understand that you can store data in variables or constants in the code.

Go by Example: Constants

So, I stored my URL constant as a string to use it in the program.

const url string = "https://www.google.com/alerts/feeds/11016768836951419989/3365315696613981974"

At this point, my understanding is that we can use gofeed to fetch our data. I took in the required snippet of code from the Github page (link above) and copied it to my program.

// My complete test.go file at this moment
package main
import (
"fmt"
"github.com/mmcdole/gofeed"
)
const url string = "https://www.google.com/alerts/feeds/11016768836951419989/3365315696613981974"
func main() {
fp := gofeed.NewParser()
feed, _ := fp.ParseURL(url)
fmt.Println(feed.Title)
}

As you can see, it is just a few simple lines of code. In my terminal (while in the path where my Go file is), I typedgo run test.go to run this program. The output I got was the title of the feed- “Google Alert — Artificial Intelligence”.

That’s awesome! We have fetched the data successfully. Now, observe the code line by line. The import part is updated automatically every time I save the program (control + s). So, no need to worry about it.

Next, we have declared a constant with the name “url”. From this, you can observe how to define the constants you want in the program.

Coming to the func main() part, the code is directly taken from the GitHub repo of gofeed. We are just changing the URL to our own feed. For now, notice the syntax. Notice that fmt.Println would print whatever we want in the terminal. The next time you want to print something, now you know how to do it.

The RSS feed has a lot of data. We want to understand how we can access the specific data we need, the title, the link to the article, and the short content about it.

Here’s where OBL comes into play once again. I tried printing out different fields and checked what data it contains. The code editor will also give you suggestions on what fields are available in the feed variable.

After observing for some time, here’s what I found-

• feed.Items contains a list of all the entries in the feed. Each entry has the details of a news article.
• The parameters of these articles which we need can be accessed with feed.Items[n].Title, feed.Items[n].Content,feed.Items[n].Link Here, n is the position of the item in the list. (Apparently, in Go, this is called a slice. Also, note that the numbering starts from 0 and not 1).

We want to have this as a function that can be reused whenever required. So, it should basically take the URL for the input and return the feed data when executed. Here’s the function I made, called fetchNews.

//Function to fetch news articles from RSS feed
func fetchNews(url string) []*gofeed.Item {
  fp := gofeed.NewParser()
  feed, _ := fp.ParseURL(url)
  return feed.Items
}
//Testing the function in main()
func main() {
  temp := fetchNews(url)
  fmt.Println(temp[2].Content) //prints content of the 3rd item
}

This is the first-ever Golang function I wrote. After testing a bit, I found that it is working as expected. The syntax on how to write functions was clearly explained in the previous resources shared. Also, the code editor itself will give you recommendations on the syntax and code completion when you are typing it.

What next?

We have achieved our objective. We can now use this function in our program whenever we want to fetch some news from any given RSS feed URL.

Also notice that using the same thought process you can literally build a similar program in any other programming language, be it Java, Python, etc. Programming is not about the language but on what you want your computer to do for you. Once you know that, it is pretty easy to find out how to do it, as we have seen.

I will explore some ideas on how we will use this function to create news posters in the next article. #StayTuned.

For more details about SLTP 2020, visit-

Self Learning Technology Program 2020

Previous articles in this series:

Creating A Project Idea That Is Right For You

Creating a basic project roadmap

Making a Trello board for better project management

Disclaimer- While I try to present the best (or the most convenient way) of what I know at the moment in building products, I want you to be aware that there can be other better ways to do something that I describe here. Always stay vigilant and keeping exploring. With that being said, if you have any suggestions, do let me know.