Top-10 most cited software engineering papers authored by Brazilian professors
Considering only papers indexed by CSIndexbr, published between 2013–2018, and using citation data from Crossref.
Crossref is an official DOI registration agency. Whenever a publisher asks Crossref for a DOI, it must provide several meta-data about the paper being registered, such as title, authors, date of publication, etc. Publishers can also provide the DOIs of each reference contained in the paper.
In this way, Crossref is creating a database of citations; given a DOI, this database stores other DOIs that cite it. More interestingly, this dataset is public (or at least part of it) and can be accessed using a REST API.
For more details about Crossref and open citations, see this interesting article, recently published at Nature News.
We used Crossref API to retrieve the number of citations of Software Engineering papers listed by CSIndexbr — a Brazilian Computer Science Index — ,which collects papers published by Brazilian professors in selected venues (journals and conferences), in the last five years (2013–2018).
Before presenting the results, we answer four important questions:
Why are you not using Google Scholar citations? Because they are "closed"; Google’s terms of service forbids data collection by bots or similar programs.
Why the number of Crossref and Google Scholar citations do not match? First, because Crossref only considers citations coming from papers with a DOI. The secod reason is because some publishers do not provide citations meta-data, as answered by them on Twitter:
As a result, Crossref citations are normally lower than the ones provided by Google Scholar. However, we also suppose there is a high correlation between both measures (although this needs to be verified).
Why are you collecting this data? Because we want to assess pros and cons of adding Crossref’s citations to CSIndexbr.
How to get the number of citations at Crossref (and therefore check the citations mentioned in this post)? You can use a url like this one (where the last part is a DOI): https://api.crossref.org/works/10.1016/j.infsof.2014.04.002. Then search for is-referenced-by field.
After this long introduction, here are the top-10 most cited software engineering papers authored by Brazilian professors (only SE papers published in 2013–2018, and indexed by CSIndexbr, ~300 papers):
- Mining questions about software energy consumption
Gustavo Pinto (UFPA); Fernando Castor (UFPE); Yu David Liu
MSR 2014
http://doi.acm.org/10.1145/2597073.2597110
Citations: 51 - Creating a shared understanding of testing culture on a social coding site
Raphael Pham; Leif Singer; Olga Liskin; Fernando Figueira Filho (UFRN); Kurt Schneider
ICSE 2013
https://doi.org/10.1109/ICSE.2013.6606557
Citations: 45 - Establishing Theoretical Minimal Sets of Mutants
Paul Ammann; Márcio Delamaro (ICMC/USP); Jeff Offutt
ICST 2014
https://doi.org/10.1109/ICST.2014.13
Citations: 41 - Automated Behavioral Testing of Refactoring Engines
Gustavo Soares; Rohit Gheyi (UFCG); Tiago Massoni (UFCG)
IEEE TSE 2013
https://doi.org/10.1109/TSE.2012.19
Citations: 38 - On the reliability of mapping studies in software engineering
Claes Wohlin; Per Runeson; Paulo Anselmo Silveira Neto; Emelie Engström; Ivan Machado (UFBA); Eduardo Almeida (UFBA)
JSS 2013
https://doi.org/10.1016/j.jss.2013.04.076
Citations: 37 - An empirically based terminology and taxonomy for global software engineering
Darja Smite; Claes Wohlin; Zane Galvina; Rafael Prikladnicki (PUC-RS)
EMSE 2014
https://doi.org/10.1007/s10664-012-9217-9
Citations: 31 - Software engineering at the speed of light: how developers stay current using twitter
Leif Singer; Fernando Figueira Filho (UFRN); Margaret-Anne D. Storey
ICSE 2014
http://doi.acm.org/10.1145/2568225.2568305
Citations: 30 - Extracting relative thresholds for source code metrics
Paloma Oliveira; Marco Tulio Valente (UFMG); Fernando Lima
CSMR/WCRE 2014
https://doi.org/10.1109/CSMR-WCRE.2014.6747177
Citations: 29 - Twenty-eight years of component-based software engineering
Tassio Vale; Ivica Crnkovic; Eduardo Almeida (UFBA); Paulo Anselmo Silveira Neto; Yguaratã Cavalcanti; Silvio Meira (UFPE)
JSS 2016
Citations: 24 - On strategies for testing software product lines: A systematic literature review
Ivan Machado (UFBA); John D. McGregor; Yguaratã Cavalcanti; Eduardo Almeida (UFBA)
https://doi.org/10.1016/j.infsof.2014.04.002
IST 2014
Citations: 24
Just a few comments about these papers:
- 5 papers are from conferences; 5 papers are from journals.
- The top-3 papers are from conferences.
- 5 papers have non-Brazilians as first authors.
- 8 papers have non-Brazilian as co-authors; 2 papers only have Brazilian authors.
- 3 papers are from 2013; 6 papers from 2014; and 1 paper is from 2016.
The following chart shows the full distribution, for ~300 papers. 182 papers have at most 5 Crossref citations; other 62 papers have between 6 and 10 citations.
