ACM TiiS Best Paper for 2017
ACM Transactions on Interactive Intelligent Systems (TiiS, pronounced “T double-eye S”) is a journal published by the Association for Computing Machinery covering research on intelligent systems that people interact with. TiiS publishes articles on research concerning the design, realization, or evaluation of interactive systems that incorporate certain form of machine intelligence. A unique aspect of TiiS, compared to other journals, is that its articles come from a wide range of research areas and communities, including human-computer interaction, machine learning, visualization, and many others. Article may focus on (a) the intelligent technology, (b) the interaction of users with the system, or ideally (c) both aspects at the same time.
Each year, the TiiS editorial board gives a best paper award to a paper published in the previous year of particular note. Nominations are collected from the editorial board and the editor-in-chief, and final decisions are made by a committee of three editorial board members. The Best Paper Award Committee members for 2017 were Shimei Pan (chair), Wai-Tat Fu, and Bamshad Mobasher.
The 2017 award went to Marius Kaminskas and Derek Bridge (University College Cork, Ireland) for their article Diversity, Serendipity, Novelty, and Coverage: A Survey and Empirical Analysis of Beyond-Accuracy Objectives in Recommender Systems, which was published in issue 7(1) of TiiS.
The committee provided the following description of their decision:
“The paper provides an extensive, thorough and well-structured literature review on beyond-accuracy quality measures such as diversity, serendipity, coverage and novelty for recommender systems. To gain insight into the relationship between these measures, a set of experiments were conducted to compare and analyze the impact of different optimizing strategies on these measures. Since beyond-accuracy quality measure is a timely topic in recommender systems research, we believe this work will not only serve as a reference point to diverse beyond-accuracy quality measures but also shed new light on the general topic of recommender systems evaluation, which would have significant impact on the design of future recommender systems in diverse context.”
Some additional context may be needed for readers unfamiliar with this area to fully understand the contributions of this paper and its selection as a best paper.
First, what is a recommender system? These are quite common, automated systems that provide recommendations to users about content, products, and services that they might like based on information that the system already has about the user and other users like them. Whenever Amazon recommends a product, Netflix recommends a show, or Medium recommends an article, the work being done behind the scenes is a recommender system.
There are many ways to evaluate a recommender system. “Accuracy” is the most common, wherein the system’s predicted ratings of products or services for a user are compared with the actual ratings provided by that user. While accuracy is certainly important, there are other factors to consider when examining the recommendations provided by a system. For example, how diverse are the set of recommendations? In some cases, such as when I’m looking for a new movie to watch, being offered some diversity in genre and period may be pleasing. I, for one, don’t need to be recommended sci-fi television series and Patrick Stewart films at every opportunity (even if those are what I keep selecting). These other dimensions of evaluation of a recommender system are referred to as “beyond-accuracy” quality measures, and they are the focus of the award winning paper.
The paper presents a comprehensive survey of the many “beyond-accuracy” measures that have been considered previously in the field. Then, through a series of offline experiments, the authors explore various optimization techniques for balancing between different beyond-accuracy measures. They also examine the correlations between various beyond-accuracy measures, and offer recommendations for how to pair different measures for improved performance.
The other two nominated Best Paper candidates for 2017 were:
Detecting Users’ Cognitive Load by Galvanic Skin Response with Affective Interference Nargess Nourbakhsh (University of Sydney and NICTA, Australia) , Fang Chen and Yang Wang (NICTA, Australia), and Rafael A. Calvo (University of Sydney, Australia)
Have You Lost the Thread? Discovering Ongoing Conversations in Scattered Dialog Blocks by Fabio Massimo Zanzotto and Lorenzo Ferrone (University of Roma Tor Vergata, Italy)
