Potential Organized Fraud in On-Going ASPLOS Reviews
Two weeks ago, I got the following unsolicited, anonymous emails about continuing potential fraud in our conferences by a small group of cheaters. Unfortunately, I have been unable to convince the email sender to reveal his/her identity to me despite promise of complete confidentiality. Therefore, I have been unable to verify that the sender is a graduate student as claimed. Still, to protect the sender’s identity, I have paraphrased his/her text to avoid unintentionally identifying him/her. After my unsuccessful attempts for about two weeks, I decided to publish the paraphrased emails hoping to stop any potential fraud in the on-going ASPLOS reviews. I simply could not remain silent while there may be on-going cheating in ASPLOS.
Email1 on Sunday, Oct 25 2020
Due to your posts, I think you are the only faculty who is interested in stopping the fraud. How about a public website listing the fraud groups? The site will reveal the groups which program chairs can then list as conflicts. I review papers for my advisor. I feel this practice is common because grad students know the details well and the review load is high. I am tired of rejecting obviously-flawed papers whose scores my advisor changes later under external coercion. I know only about my advisor’s group (group X in our community) in which a few of the top people coerce others. Person Y is a main player but there are also others who are cheating in the on-going ASPLOS reviews. There are other sources of coercion too (e.g., by a person in group Z during a physical PC meeting).
The email identifies group X, person Y, and group Z but I am unwilling to reveal any specific identities because I have been unable to verify the sender’s identity. Of course, only a small fraction of groups X and Z may be guilty, certainly not everyone in those groups. This reason is also why I am unwilling to reveal specific identities.
I replied to the sender that any anonymous posting will be ignored or the cheaters will lay low for a while and then be back. I asked the sender if he/she has any evidence for which here is the reply:
Email2 on Wednesday, Oct 28 2020
I do not have any evidence. I know only what my advisor told me verbally. Despite Huixiang Chen’s unfortunate incident, the coercion continues. In the incident’s aftermath, the cheaters are wary of discussing paper reviews via email. While the coercion was lower for Micro and HPCA, ASPLOS is worse. I contacted you because I thought revealing the groups would scare the cheaters but probably not. Could inspecting the reviews help? We all have read poor papers and asked why these papers were accepted. Sorry I don’t have anything else.
Though the sender’s identity remains unverified, please keep in mind that except for rare cases (e.g., mental illness), no grad student would falsely incriminate his/her advisor and gravely jeopardize his/her own career — very different from someone spreading lies to benefit from them or at least be unaffected by them.
What can ASPLOS do?
1. I have showed these emails to Christos Kozyrakis, ASPLOS PC Co-Chair.
2. The PC members should understand that a small fraction of the PC members may be colluding with some authors. The PC members should be vigilant and should report any unjustified reviews/scores to the PC chairs.
3. ASPLOS should discontinue online accepts which make it much easier to collude quietly. Having every paper be discussed at the PC meeting sets the bar for the reviewers of every accepted paper to justify the acceptance to the entire PC. Though far from perfect, this bar is better than online accepts (without online accepts, the same number of papers can be discussed at the PC meeting as with online accepts, but every accepted paper would be vetted by the entire PC).
4. The PC chairs can analyze the review data in HotCRP to find review cliques (A accepts B’s paper who accepts C’s paper who accepts A’s paper) where the other, non-clique reviewers disagree with the accepts. The PC members can help by flagging unjustified reviews.
5. In my previous post, I noted that when another SIG had a similar collusion problem, the SIG chair personally analyzed several years of conference data and caught the cheaters. A single official can figure this out from the inside much easier than any of us trying from the outside.
If any of you have any evidence or information, please email me at my secure account tnvijayk@protonmail.com (for your protection). I guarantee full confidentiality at my end.
We must stop this scourge before it demolishes our research community and conferences. Paper acceptance in these conferences plays an enormous role in (a) which grad students get coveted jobs and (b) which faculty and researchers get tenured/promoted, rewarded, awarded and respected. Most importantly, on the human side, the graduate students in group X are getting crushed between their advisors and their own conscience. I am gravely concerned that some of the students may follow Huixiang Chen’s unfortunate example. There is too much at stake apart from the fundamental notion of fairness.
In addition to conference reviews, proposal reviews may also be affected. After my previous post, I got a short, unsolicited, anonymous email paraphrased below (I notified NSF right away):
More serious than conference papers, there is rigging of proposal reviews via clique bids and super-high scores.
I replied asking for evidence, but I got no reply. In NSF’s proposal review process, each panelist is invited to bid for the proposals he/she wishes to review. Thus, the process is susceptible to such organized fraud. Please notify NSF and other agencies if you have evidence or information.
About the Author: T. N. Vijaykumar is Professor of Electrical and Computer Engineering at Purdue University and works on computer architecture.
Disclaimer: The opinions expressed in this post are the author’s and not of Purdue University.
