How people come to Nallur

by Keshan de Silva and Yudhanjaya Wijeratne

Every so often we come across festivals that profoundly change how people think, act and behave. Christmas is perhaps the most universal example: others, perhaps more localized, range from Hanukkah to Diwali to Ramadan.

One such festival happens at the Nallu Kandaswamy Kovil. Every year, in the months of August and September, hundreds of thousands gather in Jaffna, Sri Lanka, in the town that was once the heart of the Jaffna Kingdom. They come for the rituals of Murugan, the many-faced Hindu god of war, a philosopher-warrior who destroys and inspires at the same time. Certain ceremonies, which have not been altered for centuries, are performed: Murugan’s spear and idols are paraded around the temple grounds in magnificent chariots, and the ceremony ends with a re-enactment of the Holy Wedding between the god and his consorts.

At LIRNEasia’s Big Data Unit, we grew interested in the Nallur festival — especially since it ended quite recently. Thousands of devotees celebrate the occasion, many of them coming to Jaffna to celebrate. Could we see these temporary population changes reflected in data? And if we could, could we do a Cotton-eyed Joe and identify where these devotees came from, and where they went?

Knowing where people come from helps governments and organizers plan ahead and predict what the next event will be like. In this case, private buses are a common form of transport to the Northern Province: perhaps more of such transport needs to be arranged between one particular region and Nallur. Perhaps, in the case of large, recurring, events of national scale and import, there are road networks and essential services that can be strengthened along possible routes and hotspots.

Finding the ingredients

Our problem requires several pieces of data — we need to know how many people there were in the Nallur area before the festival starts, and how many there are once the festival gets under way. We also need to know which region the newcomers originated from.

In a perfect world (from a research perspective), everyone would have GPS devices, and we’d have access to that data. But even if you discount the disturbing privacy implications, we simply don’t have GPS data like that.

Instead, we used the next best thing: Call Detail Records (CDR for short). These records are generated every time a call is made on a cellular network. Each record contains the timestamp of the call and the two cell towers that were connected in the process of delivering the call.

Given that Sri Lanka‘s extremely high mobile subscription rate (over 100% of the population — meaning a sizeable portion of the country actually has two or more SIMS), CDR could answer each of the problems we were facing.

We happened to have a dataset of these records for 2013 from all major telecommunications providers. The data was anonymized, because of privacy concerned, until individual users for no longer trackable.

We could estimate the number of people in Jaffna before the festival by counting subscribers using the appropriate base towers: we could estimate the population increase by estimating the increase in communications using the Nallur towers: and, by checking backwards and seeing how subscribers were distributed before the festival, we could identify where people were coming from. After all, if we know which tower people are using, we know roughly where they are.

And so, armed with the anonymized data, we set to work.

Tasting the data

The first question was: could the dataset that we had support our analysis?

To test this, we first filtered the raw dataset for CDRs originating from 2013-06-01 to 2013-10-31. This gave us the people who were in the region in the timeframe of the festival.

Next, we ran analysis to find the cell ID that each of these people frequent, using the tower of origin, and filtered that by the Jaffna District, where the town of Nallur is in.

This allowed us to estimate the population of the Jaffna District and compare that against time to see the influx during the festival. It also neatly prevents one subscriber from being counted to more than one cell, which makes these as close to unique people as we can get.

As expected, there was a population increase that showed up very clearly in the analysis, as shown below.

So, yes: CDR data tentatively shows us that the population of increased during the festival time, from 12th August to 5th September.* How many of these people were visitors?

We can extrapolate this by plotting the average number of visitors daily against the population of the region for a time period surrounding the festival. We can see that visitors for July are quite low — most of the time it’s below the average. It shoots up around festival time and dips again. On October 13th, the President of Sri Lanka officially opened the Colombo-Jaffna rail line — an event of phenomenal significance, given that the line had been shut down due to many years of war — and the visitor population skyrockets again.

There’s also a sharp drop in population in the 13th and 14th of August. This happened throughout the CDR dataset regardless of location, and could be caused by a failure in the mobile operator(s) or a power failure throughout the country that took a lot of phones offline. There was also peak in the graph from 15th September to 20th September: both 15th September (Tuesday) and 18th September (Friday) were public holidays in 2013, which makes a long weekend on that week.

Once these anomalies are accounted for, the visitor population seems to give us a reasonable estimate of how many came to Jaffna district for the Nallur festival.

The first analysis: where devotion comes from

We mentioned earlier that we can analyse the dataset for the cell IDs that subscribers frequent. Now that we’ve identified the influx to Jaffna, we can look at a larger time period to identify where this entire visitor population came from:

When we lay this out using a heatmap, we can quite easily identify where people come from:

We can see straight away that most people come from within the Jaffna region itself. That’s to be expected. After all, we are basing our analysis on this region.

When we move away, we see numbers of people coming from Colombo; less from Trincomalee and Batticaloa; smaller pockets from Kandy and Hatton; and a general scattering of people from all across major towns and city areas across the island, even from the extreme South.

A good way of determining who the vistors really are is to identify the percentage in visitors to the Jaffna district from a given region.

Here’s what this granular look yields when projected onto a map:

The second analysis: where devotion goes to

We don’t have to limit ourselves to just where people came from. If we look back at our festival dataset, we can identify where people were the most during the festival.

The largest number of people went to Jaffna — the town this time, not the district (it’s a bit confusing — a lot like New York State and New York City). Most generally stuck within the orbit of Jaffna and Nallur, which is where festival happened. A very small percentage went further afield and checked out islands (Delft in particular makes for an excellent vacation in Jaffna, with an isolated beach strip).

This analysis, mapped out against time, yields some interesting insight. More people flock to Nallur towards the latter parts of the festival.

The peak of the Nallur festival is generally acknowledged to be Ther — to quote roughguides.com, Ther is “when an enormous chariot is pulled around the town by huge crowds of sarong-clad men; on the following day, particularly enthusiastic devotees mortify themselves by driving skewers through their bodies in honour of the god and making their way to the shrine accompanied by drumming and piping, stopping periodically to dance en route. Even more extraordinary are the devotees who, using skewers driven through their backs, suspend themselves from poles. These poles are then attached to the front of trucks and tractors, and the devotees are driven through town to the temple, dangling in front of their vehicle like bait on a fishing line.”

But Ther falls on the 24th of October; the increase in numbers happens towards the very end — towards the date of the Divine Wedding.

That’s a very interesting point to think about. There’s not a lot of concrete information on visitor data out there, and Ther being the highlight is largely anecdotal: we can, with this kind of analysis, figure out what the most popular events of the festival are, and claim it with more certainty than an anecdote.

Lessons from Nallur, and what we can cook up next

So what of Nallur? It’s up to the government and the organizers to understand and apply these insights. Perhaps Kandy needs more transport arranged: perhaps the town of Jaffna might need more bare space next time to accommodate everyone. Perhaps a tourism industry can be encouraged around events like these. There are certainly business opportunities that can be encouraged — for example, all those people coming to watch definitely need places to stay, especially in those last, unexpectedly popular days.

Great events happen all across the world for all kinds of reasons — consider, for example, the Kandy Perehara, a T20 cricket match, or a May Day rally — or even something as exotic as Burning Man. Nallur is a case study, but is not the limit of this kind of analysis. With Call Data Records, we can figure out where people flock to, and where they come from when these events happen. Like what we saw here, you do find unexpected insights in these things, and the solutions are diverse.

Imagine what else we could do next.