Creating and hosting an HackerNews mirror on Netlify for free

6 min readJan 26, 2020

In this short tutorial I will illustrate the steps and solutions that I used to make a HackerNews mirror: hacker-news.news.
I call it mirror, as instead of simply being a clone of HN iterates directly with the original backend, replicating the original functionalities and adding new ones. It could also simply be seen as an interface.

The idea behind this project is to create a more attractive interface for HackerNews and to add some features that I personally find missing from the original site, such as previewing articles when scrolling through the feed. The original site design is minimal, which is absolutely not a defect and makes it fast and easy to use. In this tutorial, however, we will see how to create a more modern interface that replicates its functionality.

The steps we will follow to create the platform are:

Choice of a template for the interface
Integration of the template into an Angular project
Integration with the HackerNews API
Add the search option with Algolia
Deploy on Netlify
Mirror functionalities in HackerNews
Netlify Lambda functions for Login / Comments / Upvote / Downvote
Adding a leading image and a description to the story
Caching with Firestore

The code of the project is available here: https://github.com/GabrielePicco/hacker-news-rich

1. Choice of a template for the interface

For graphics you can use any template or directly Angular Material.
My choice for this project was to use this template: https://wrapbootstrap.com/theme/material-style-material-design
(the only single expense that can be avoided by using a free template).

2. Integration of the template into an Angular project

For the creation of the web application I chose Angular, as I like the modularisation and separation it provides between html, css and typescript code.

You can follow the official setup tutorial and create a new application with:

ng new hacker-news-mirror

and then create components with:

ng generate component component-name

The generated component will correspond to a folder containing 4 files:

one for the html code, one for the css, one for typescript and one for the unit test.

3. Integration with the HackerNews API

Each component must be able to interact with the HackerNews backend. For example, if we want to view the top list of articles, we must be able to access an endpoint that provides us with the desired information.

Fortunately, HN provides an API that is sufficient for all basic operations, such as retrieving articles, posts, authors, etc.

For example we can simply get all the informations for the story 22147892 calling the endpoint: hacker-news.firebaseio.com/v0/item/22147892.json

{
  "by" : "fakeittillyoumakeit",
  "descendants" : 7111,
  "id" : 22147892,
  "kids" : [ 8952, 9224, 8917],
  "score" : 1000001,
  "time" : 1579978501,
  "title" : "Hacker News Mirror (With Preview)",
  "type" : "story",
  "url" : "https://hacker-news.news/TopStory"
}

using the API we can easily create an angular service capable of obtaining stories in various categories, authors, etc.

4. Add the search option with Algolia

We can also easily integrate the APIs provided by Algolia to add a search functionality within the platform.

For example the endpoint hn.algolia.com/api/v1/search?query=apollo
will simply return a list of stories related to “apollo”.

As previously done, an Angular service can easily be created to add search functionality to the platform.

5. Deploy on Netlify

Several tutorials illustrate how to deploy a website via Netlify.
In short, simply connect to a repository and configure options to build.

Netlify will take care of the whole process and update the distributed version with each update on the master branch.

If you want to try clone the project repository and connect it to your Netlify account.

6. Mirror functionalities in HackerNews

One of the most interesting parts of the project was the integration of the functionalities for logging in, posting comments, submitting new stories, upvoting and downvoting.

To make the platform really useful, all the actions performed must be replicated on the original platform, so that if I add a comment, it will also be visible to everyone who uses the HN site.

With a very brief analysis of the original interface of HackerNews it is noted that all the functions are available on the base address of the site, for example the login form sends the data to the address news.ycombinator.com/login

In addition, all the features seem to work if called by adding the password and username as body parameters.

For example it is possible to add a comment with a call similar to the following one:

const body = new URLSearchParams();
body.set('acct', credentials.username);
body.set('pw', credentials.password);
body.set('parent', parentId);
body.set('text', text);return this.http.post('news.ycombinator.com/comment',body.toString(), {
  headers: headers,
  responseType: 'text',
  withCredentials: true
}).subscribe();

A first attempt was to integrate the code to call the endpoint into Angular by directly adding the username and password in the body of the request, but without success.

The main problems here were the Cross Origin which blocked the request and the fact that a secure token (therefore not erasable by Angular) was saved in the domain from which the endpoint for login was called.

After a few attempts the solution was to use the lambda functions of netlify which I will illustrate in the next section.

7. Netlify Lambda functions for Login / Comments / Upvote / Downvote

The idea of using the Netlify lambda functions was therefore to insert a sort of forwarding that avoided the cross orgin and the storage of the HN token in our domain.

Lambda functions are quite simple to use, just insert the code (NodeJs) of the functions in a folder and specify the path in the Netlify interface

The solution adopted for the login was therefore:

The login interface calls a lambda function by passing username and password as parameters
The lambda function calls the original HN endpoint and returns the success or failure result
If successful, the credentials are stored in a token, encrypting the password with a combination of personal fields.

So now we have the correct credentials saved in the token and we can use them to call the other features, always using the same redirection mechanism with the lambda functions.

It is therefore now possible to develop an Angular service that calls the lambda functions for the various functions (comments, submission, upvotes …)

8. Adding a leading image and a description to the story

Now we can add, using the Mercury library, information to stories such as the summary of the article and the main image.

since Mercury is a NodeJs library we can simply add a lambda function to get the information by passing it a link as a parameter.

const Mercury = require('@postlight/mercury-parser');

exports.handler = (event, context, callback) => {

  Mercury.parse(event.queryStringParameters.url)
    .then(result => {
      callback(null, {
        statusCode: 200,
        body: JSON.stringify(result),
      });
    })
    .catch((err) => {
      callback(err);
    });
};

The function allows us to retrieve the necessary information to be shown in our interface:

9. Caching with Firestore

Since we have not yet integrated enough APIs into our platform, it is possible to add a caching level to prevent the Mercury library from parsing the same link multiple times.

To do this we create a collection in Firestore:

and we add the control that the article is parsed only if it is not already present in our collection.

Conclusion

By integrating different services, libraries and APIs, it was possible to create a platform capable of extending the functionality of HackerNews, while maintaining the possibility of interacting with the original site. Furthermore, through Netllify and the lamda functions it has been shown how the platform can be published completely free of charge.

The lambda, netlify and cloud firestore functions have a rather generous free plan and are perfectly scalable. If the platform had huge traffic, it would be able to support it, turning my free plans into a high cost. So I hope that not too many people read this article, that those who read it find it useful and that I have enjoyed writing it.