Let Users Browse Your Website with Simple, Data-Driven Lists
Whether you are building the next Amazon, Product Hunt or Hacker News, at some point you will come up with a challenge of building lists that will make it easier for users to find the most relevant items in your vast repository of products, startups or articles.
In this tutorial, I’ll show you how to easily create data-driven lists of:
- the most-liked items
- recently liked items
- items that have the best ratio of likes to views
You can see the lists in action here.
You won’t need a server or a database or write any backend code to create the lists — just a basic knowledge of JavaScript and a DataDrivenJS account. The approach described here will work for all websites regardless of the development stack behind it.
If you are not familiar with DataDrivenJS, think of it as a web analytics tool but for developers. It allows you to track — just like Google Analytics —the behaviour of your users and then access the tracked data with just a few lines of JavaScript code.
FYI: This is the second part of a larger tutorial and in the first part we have covered how to: 1) allow users to like items, 2) show how many times each item was liked and viewed, 3) order the existing list of items by likes and views. I highly recommend reading and completing it first.
0. Prerequisites
Before you continue with the tutorial, you should:
- sign up for a free account and create a project
- download the source files of the tutorial from GitHub. The code in the “Letters-1” folder contains all the changes implemented as explained in the first part of this tutorial — we will use “Letters-1” as a starting point for this part of the tutorial (you can see it working here). The “Letters-2" folder contains the final version with our data-driven lists.
- before you start, you will need to replace the DD.js script tag (it’s in the head tag) in all HTML files from the “Letters-1” folder with your project-specific tag — you will get it once you create your own project in DataDrivenJS. Otherwise you would work with a project you have no access to.
2. The Plan
We need to create these three lists on the home page:
- recently liked items
- the most-liked items
- items that have the best ratio of likes to views
To complete the task we need to:
- set up the tracking of views and likes on the item.html page
- create three publicly accessible data feeds containing the top four letters for each list and load them on the index.html page
- use the data from the feeds to display the lists
- schedule how often the data feeds — and therefore the letters on the list — should be refreshed
Tracking Views and Likes
First, let’s see what data is already being tracked on our website.
Sign in to the DataDrivenJS Console, select your project and click the Dev Mode button in the top right corner. You will be asked to provide a URL — enter the URL pointing to the index.html file.
Tip: if you are using Chrome browser then you can use a URL that starts with file://… All other browsers will require a running localhost to work properly.
Now, open the browser’s console — all the datapoints tracked by DD.js are logged there if you are in the Dev Mode.
There is no custom tracking set up on index.html — the datapoints you see in the log are tracked automatically for any visited page. Some of them, for example, the landingURL, are tracked just once, when a new session starts. Others, like URL, are tracked every time a new page is loaded.
Now, navigate to any of the letters (item.html).
Note: if you haven’t completed the first part of the tutorial and you are working with the sources from the ‘Letters-1’ folder, at this point DD.js will try to recreate missing data feeds and they will likely be empty — you can safely ignore them.
Click the like button and another event will show up in the log:
The tracking of the liked metaevent had to be manually implemented in the code. The implemenation was one of the steps of the first part of the tutorial and was required to displayed how many users liked each letter. Let’s see how it was done.
Open item.js file from “Letters-1” folder and find the click listener in the addLikeBtn method. When a user clicks the like button an event is sent to the DataDrivenJS:
// use global letterID to track the liked letter
DD.tracker.trackMetaEvent('liked',letterID);The letterID, for example “d-green-round-serif-bold-uppercase”, informs us which letter was liked. Its value is retrieved from the page URL:
https://datadrivenjs.github.io/the-letters-tutorial/letters/letters-1/item.html?letter=d-green-round-serif-bold-uppercaseWe also implemented the page view stats in the first part of the tutorial, but, instead of tracking a custom metaevent, we used automatically trackedURL metaevent.
Now, however, one of the lists we are going to create will show letters that had the best ratio of likes to views and we need to be able to exactly match the values of the corresponding events. The URL event will be of no use to us, instead, we need to track the letterID as an additional viewed metaevent:
DD.tracker.trackMetaEvent('viewed',letterID);Add the above tracking code right under the setTheMainLetter method call — it will be executed just once, right after a page is loaded:
var letterID = getLetterID();
if (letterID){
setTheMainLetter(letterID);
// track views of the letter in the same format as 'liked'
DD.tracker.trackMetaEvent('viewed',letterID);
}else{
console.warn('No letter ID found in the URL.');
}Save the file, reload the item.html page (you will still be in the Dev Mode) and like the letter again. This time, the browser’s console log should contain, among others, viewed and liked metaevents:
Tracking Some Dummy Data
Before we start implementing the lists, let’s track a couple of visits to have some dummy data to work with. Go around the website, view some letters, like a couple of them. Once you’re done, run this code in the browser’s console to end your visit:
DD.tracker.stop();You won’t need to do that in the production environment, however, forcing a visit to stop will save you a lot of time while you are developing. Otherwise, you would have to wait around 20 minutes for a visit to close due to the lack of activity.
Tip: To see what data was collected in your project, select Tracked Data from the hamburger menu in the top right corner in DD Console.
Now, to the fun part! Let’s read the tracked data to create our lists.
3. The Most-Liked Letters
Reading the data is quite straightforward. Let’s start with getting the most-liked letters. In the index.js file, we need to add the following code:
var getMostLikedLetters = function(){
// create data feed, it has to be named so you can manage it
// in the DD Console var feed = DD.data.feed('The Most-Liked Letters'); // select the last 4 unique values of 'liked' meta event feed.select(
DD.data.datapoints.metaevent('liked'),
DD.data.datapoints.metaevent('liked')
.count().as('likes')
).orderBy(
DD.data.feedColumn('likes').desc()
).limit(4); // read the entire data feed: DD.reader.read(feed, {}, function(response){
addList('#mostLikedLetters', response.results);
});
}
Let’s break the select down. First we select all values (letterIDs) of the liked event:
feed.select(
DD.data.datapoints.metaevent('liked'),
...and then we count how many times each value was tracked:
feed.select(
DD.data.datapoints.metaevent('liked'),
DD.data.datapoints.metaevent('liked')
.count().as('likes')Note: if one of the selected columns is using an aggregate function the results are automatically grouped by the remaining columns, in this case by the value of the liked metaevent.
We give this column a friendly alias:
feed.select(
DD.data.datapoints.metaevent('viewed'),
DD.data.datapoints.metaevent('viewed')
.count().as('likes')so we can easily refer to the column and sort the results in descending order:
feed.select(
DD.data.datapoints.metaevent('liked'),
DD.data.datapoints.metaevent('liked')
.count().as('likes')
).orderBy(
DD.data.feedColumn('likes').desc()
)Note: the select method accepts datapoints and turns them into feedColumns — when you order the results make sure you use the latter.
Finally, we limit the results to just 4 items:
feed.select(
DD.data.datapoints.metaevent('liked'),
DD.data.datapoints.metaevent('liked')
.count().as('likes')
).orderBy(
DD.data.feedColumn('likes').desc()
).limit(4);The select itself doesn’t return any data — it only serves as a schema of a data feed. To request the data from the feed you use DD.reader.read — the first argument is our data feed object, the second one is an optional subquery, and the third one is a callback function:
DD.reader.read(feed, {}, function(response){
addList('#mostLikedLetters', response.results);
});TheaddList method isn’t part of the DD.js API, but of the site work on. It accepts a selector of an element to which you want to add the list and the data feed results:
DD.reader.read(feed, {}, function(response){
addList('#recentlyLikedLetters',response.results);
})Now, let’s try it:
getMostLikedLetters();The first request to read a feed will take longer as the feed needs to be created. The subsequent requests will return data instantly.
Important: a reminder for those who haven’t completed the first part of the tutorial. If you receive the following warning: “This query is trying to access data from a public data feed that doesn’t exist yet” then sign in do DD Console, open the Dev Mode from there and run your query again. This time the feed will be created.
Auto-refeshing Data Feeds
If everything is working correctly, we need to schedule how often the data feed should be refreshed. Depending on your needs you can refresh it “every hour” or “every Monday at 7 am”.
Tip: If you are interested in the real-time access then make sure to follow us on Medium — we’ll write a separate article about it.
Switch the browser tabs to the DD Console — there’s a button that will open the scheduler right above the table with the results. Set up the scheduler to update results every hour.
Now, let’s get the other lists done.
4. The List of Recently Liked Letters
To create the list of the recently liked letters we need to sort our feed by timestamps:
var getRecentlyLikedLetters = function(){
// create data feed, it has to be named so you can manage it
// in the DD Console var feed = DD.data.feed('Recently Liked Letters'); // select the last 5 unique values of 'viewed' meta event feed.select(
DD.data.datapoints.metaevent('liked'),
DD.data.datapoints.metaevent('liked')
.timestamp().max().as('lastTimestamp')
).orderBy(
DD.data.feedColumn('lastTimestamp').desc()
).limit(4); // read the entire data feed: DD.reader.read(feed, {}, function(response){
addList('#recentlyLikedLetters',response.results);
})
}getRecentlyLikedLetters();
Instead of counting the values, this time we select event timestamps which we will use to sort the results:
feed.select(
DD.data.datapoints.metaevent('liked'),
DD.data.datapoints.metaevent('liked')
.timestamp().max().as('lastTimestamp')To avoid a situation in which the last 4 results will contain the same letter multiple times, we use the max() aggregate function to get only the last timestamp per value:
feed.select(
DD.data.datapoints.metaevent('viewed'),
DD.data.datapoints.metaevent('viewed')
.timestamp().max().as('lastTimestamp')I think the rest of the code is self-explanatory, isn’t it? Again, if everything works as supposed, remember to schedule the autorefreshing of the data feed.
5. The Letters with the Best Likes to Views Ratio
This is a bit more complex task as we need to calculate the ratio of likes to views. Let’s see what will happen if we try to select likes and views into the same data feed:
var getBestConvertingLetters = function(){
// create data feed, it has to be named so you can manage it
// in the DD Console var feed = DD.data.feed('The Letters with the Best Likes to Views Ratio'); // select and count all values of 'viewed' and 'liked' meta events feed.select(
DD.data.datapoints.metaevent('viewed'),
DD.data.datapoints.metaevent('viewed')
.count().as('views'),
DD.data.datapoints.metaevent('liked'),
DD.data.datapoints.metaevent('liked')
.count().as('likes')
); // read the entire data feed: DD.reader.read(feed, {}, function(response){
console.log(response.results);
})
}getBestConvertingLetters();
If you take a look at the results in the DD Console you will notice that the number of likes and views of the same letter are stored in separate rows:
We could use JavaScript to match the data for each letter and calculate the ratio of likes to views. However, that would require us to load stats for all the letters. It may not be a problem when you work on a tutorial, but in the real life your feed may contain thousands of items and there’s no point in loading all of them if you need just the top four items.
First, we need to group the rows and we can achieve it by merging the viewed and liked columns:
feed.select(
DD.data.datapoints.metaevent('viewed'),
DD.data.datapoints.metaevent('viewed')
.count().as('views'),
DD.data.datapoints.metaevent('liked'),
DD.data.datapoints.metaevent('liked')
.count().as('likes')
).mergeOn(
DD.data.feedColumn('viewed'),
DD.data.feedColumn('liked')
);Check the results of the above query in DD Console, we’re almost there!
Now, that each letterID has corresponding likes and views, we can add an extra column with ratio of the two and use it to sort and limit the results:
feed.select(
DD.data.datapoints.metaevent('viewed'),
DD.data.datapoints.metaevent('viewed')
.count().as('views'),
DD.data.datapoints.metaevent('liked'),
DD.data.datapoints.metaevent('liked')
.count().as('likes')
).mergeOn(
DD.data.feedColumn('viewed'),
DD.data.feedColumn('liked')
).addColumns(
DD.data.formula().divide(
DD.data.feedColumn('likes'),
DD.data.feedColumn('views')
).as('ratio')
).orderBy(
DD.data.feedColumn('ratio').desc()
).limit(4);Finally, our data feed has exactly the data we need:
Here’s our final code:
var getBestConvertingLetters = function(){
// create data feed, it has to be named so you can manage it
// in the DD Console var feed = DD.data.feed('The Letters with the Best Likes to Views Ratio'); // select and count all values of 'viewed' and 'liked' meta events feed.select(
DD.data.datapoints.metaevent('viewed'),
DD.data.datapoints.metaevent('viewed')
.count().as('views'),
DD.data.datapoints.metaevent('liked'),
DD.data.datapoints.metaevent('liked')
.count().as('likes')
).mergeOn(
DD.data.feedColumn('viewed'),
DD.data.feedColumn('liked')
).addColumns(
DD.data.formula().divide(
DD.data.feedColumn('likes'),
DD.data.feedColumn('views')
).as('ratio')
).orderBy(
DD.data.feedColumn('ratio').desc()
).limit(4); // read the entire data feed:DD.reader.read(feed, {}, function(response){
addList('#bestConvertingLetters', response.results);
})
}
Let’s run it:
getBestConvertingLetters();Tadaam, all three lists are ready!
If you still have time, let’s quickly go through some tips how to further tweak our lists.
Using Segments
Since we have implemented the tracking of theviewed metaevent after the liked it may happen that the ratio will be higher than 1. You could mitigate this issue by selecting the data from the visits tracked after a certain point in time or from the visits containing theviewed metaevent (see “Changing the timeframes” below) or you could use data from a segment of visits in which a viewed metaevent was tracked at least once, regardless of its value:
// segment of visits with at least one viewed metaevent
var visits = DD.data.segment('Visits with a Viewed Event').where(
DD.data.datapoints.metaevent('viewed')
);The created segment definition needs to be added to the feed definition using from method:
var visits = DD.data.segement('Visits with a Viewed Event').where(
DD.data.datapoints.metaevent('viewed')
);feed.select(
...
).from(
visits
);
Removing Infrequent Events from the List
The data feed with the “best-converting” letters is sorted by the ratio of likes to views. This means that an item viewed once and liked once (ratio = 1) will show up higher on the list than another item with 100 views and 99 likes (ratio = 0.99). In order to avoid such situations we can eliminate from the feed all the letters that had less than 10 views. We do that using having method:
feed.select(
...
).having(
DD.data.feedColumn('views').isGreaterThanOrEqualTo(10)
).from(
visits
);Changing the Timeframes
The results in the first feed are ordered by the timestamp so it will always return the latest results. But what if we would like to show the most-liked and the best-converting letters today or over the last week?
You need to update the feed definition to select data from a segment (a subset) of visits:
var visits = DD.data.segment('Last Week').startedAfter(
'Last Week'
);feed.select(
...
).from( visits );
That’s it!
If you want to build data-driven lists, recommendations or personalisation features then DataDrivenJS can save you lots of time and troubles so why don’t you take it for a ride? It’s free after all.
