How R and Shiny Helped Predict Turkish Referendum
Recently Turkey held a referendum on changing its constitution and its type of government by handing more power to the president from the prime minister and the parliament. Official results are yet to be announced, but the legislation passed with 51.4% of the votes (51.2% domestic, 59.5% overseas see the Wikipedia article for more detail). That’s important news but that’s not the news. Our R/Shiny tool predicted the referendum as 50.6% Yes (Evet).
You can try it here. It’s in Turkish (sorry), though easily understandable. About 3.5k people used the tool in the first 10 days, they produced about 12k scenarios yielding to the aggregate result of 50.6%. Here is the full story behind it.
Some Background
I had an R package called secimler (elections in Turkish) that can be installed via GitHub (see here). The package consists of election results on a ballot basis for the last two general elections in Turkey; November 1, 2015 and June 7, 2015 . It is mainly used for educational purposes to teach data analysis and automated reporting (mainly through tidyverse and Rmarkdown). One day, I thought I can also use Shiny in my works (so far I disregarded it as unnecessary). I started investigating for a side project. It is how referendum scenario analysis was born.
One day, I thought I can also use Shiny in my works. I started investigating for a side project. It is how referendum scenario analysis was born.
Though, instead of a simple polling app I wanted a different approach. I had the data from the previous elections. I knew about the significant tendencies. Therefore, I decided to ask what they think about four important factors which could affect the outcome of the referendum, then put their opinions into a model and yield a yes/no (evet/hayır) prediction.
I asked an experienced friend, Nezih, to help manage the project and handle the UX design. He also helped greatly to promote the tool. He did his best, but there were limits on Shiny and my proficiency in Shiny.
Building the Underlying Model
The four factors are the tendency of undecided, the tendency of MHP voters (MHP is a right-wing political party, which the leadership decided to support the constitution change though many followers are skeptical of such change), success of CHP’s campaign (CHP is the main opposition party) and the attendence of HDP voters (HDP is the fourth significant political party mainly focused on the issues of the largest minority group in Turkey, Kurds).
Apparently I was on the right track. Right after the election Ipsos held a similar research and published on Hurriyet & CNN Turk (news outlets).
I had the data from the last two elections, so I could build a numeric model to spit out predictions not only for the whole country but also individual cities (though, most of the interest was on country-wise predictions).
The model I built was some ad-hoc model with lots of “educated guesses”. Admittedly, it had no scientific foundations (no accepted prediction model in the literature is used, as if they would work). A model is only as good as its assumptions, and I believe I made some good ones which brought the increased accuracy.
I assumed vote swings between the previous two elections as the undecided as there were only 6 months between those two elections with a significant difference (btw, much happened since the last election). I also introduced some bias in the results for two reasons: First, my audience was not homogeneous or necessarily objective. Second, there might indeed be some “bias” in the vote counting. So skewing the input metrics (e.g. mid-point is not actually a mid-point) a bit helped the model to achieve better accuracy.
I will add the calculations and the Shiny codes to secimler package with the collected data from the users, so you will be able replicate the results.
Building the App
Building the app on Shiny was fairly easy. It took me half a day to get the basics and build a working prototype and 2 more days to do fine tuning to finish. The outcome was a marvelous (and responsive) website built with only R!
The outcome was a marvelous (and responsive) website built with only R!
Shiny is a great tool to rapidly build a product. Though, it is not easily customizable or it is not great on production ready peripheral tools. I had to hack my way into changing slider labels, I had to hardcode some tracking tools (Google Analytics and Facebook Pixel) and I had to manually define “loading” UI elements. Little but nice details. I hosted it on Shinyapps.io, which turned out to be a reliable service. I used Google Drive for data storage.
Nezih provided amazing help in testing the app and guiding the UI and UX of the project. He made the interface much more understandable and intuitive. Without him, this project would be much less effective.
Reception and Results
We promoted the app through our Facebook, Linkedin and Twitter accounts. I also spent some money on some fine arranged Facebook advertising with quite a modest budget (~100 TL / 35$). Thanks to Nezih’s efforts, we had two articles on Journo (the introduction and the analysis). About 3.5k people used the tool (according to GA data) and they made about 12k scenarios. So I cut the advertising at quarter of the budget :) Here is the scenario distribution.
Incidentally, our error rate (0.8% or 0.6% if you just count domestic votes) is quite low compared to other polling firms predicting the referendum results. Here are some of their latest predictions .
- ORC (59.4% Yes / 8% error)
- Qriously (59% Yes / 7.6% error)
- A & G (60.8% Yes / 9.4% error)
- THEMIS (46.1% Yes / 5.3% error)
- KONDA (51.5% Yes / 0.1% error) Congratulations!
- AKAM (46.2% Yes / 5.2% error)
- MAK (56.5% Yes / 5.1% error)
- ANAR (52% Yes / 0.6% error) Yay!
- Gezici (51.3% Yes / 0.3% error) Yay!
- Konsensus (48.8% Yes / 2.6% error)
- Sonar (48.8% Yes / 2.6% error)
- CHP (42.3% Yes / 9.1% error)
- Politic’s (55.7% Yes / 4.3% error)
- NET (48.9% Yes / 2.5% error)
Conclusions
We reached following conclusions with our little experiment.
- R, again, proved to be a great programming language for data tidying and analysis.
- Shiny is a powerful tool for interactive data products. You can build a web tool in almost no time and do something with it.
- Data based predictions can increase the accuracy of predictions. Model building is equally important.
- Turkey’s politics is a good learning opportunity.
