Psychology of a Reviewer: How to Advertise yourself and Satisify your Ego

I have been challenged for awhile by a or a group of reviewers who I assume, form a circle of academic ‘friendship’. The main challenge is to ‘cite’ their suggested studies. Of course all these studies are of studies of these researchers from the circle! Probably, this post will unavoidably affect my being published in ‘certain journals’, but…

That is OK and totally fine ‘if’ these studies are somehow related to mine. Referring to some other studies is actually what it is supposed to be, right? Since I quoted a body of others and clearly stated ‘my study is based on an other researcher’s originial study. I simply aimed to extend it and apply a different approach. So here is the results;

“It’s not enough to use a hot and sexy new regression method (which of course I applaud in and of itself) when, in all other converging ways — coverage, theory and empirical methods — this ms fails over and over again.”

This is how the reviewer responded to my study. Hot and sexy? It is basically regression analysis yet with Bayesian Framework. No one claimed it is something new or hot and sexy, yet it is simply another way of handling the statistics.

Here is the reviewer’s comment on, let’s say journal X:

“There is no mention of corpus-based research on alternations in L2/EFL learners by the two main groups in this field, namely Ellis, Roemer and Wulff or Deshors, Gries and again Wulff, who have studied the genitive alternation…”

Here is the (another) reviewer’s comment on, let’s say journal Y:

“There are also many more studies on alternation phenomena in EFL learner language (for example, see the work by S. Wulff, S. Gries, B. Szmercsanyia; there are more!)” “Finally, there is no mention of much of the central corpus-based work on syntactic priming, which has studied priming in corpus data, quantified verb-alternant preferences, proposed moderators of priming effects; much work by Jaeger comes to mind (alone and with Snider), plus a bit of work by Gries and Szmrecsanyi, and again this predictor is not included in the model.”

Also comment after revision for journal Y

“The literature review also lacks in representing current research in the study of alternation phenomena based on learner corpora.”.

Well, after feeling curious, just simple Google check verifed relation the of “alternation phenomenon” and the reviewer. Anyways…Problem is, I did not focus on “alternation phenomenon”, the focus is on dative constructions, which naturally have two alternating forms in English.

Again suggestion for citing…

Instead, we are offered the assessment that a doctoral dissertation by three authors (?) is ” the most comprehensive study utilizing learner writing in the second language acquisition/learning environment”. The way in which this paper appears to toot its own horn (I am guessing, I do of course not know who the present authors are) and pretends that all these strands of work do not exist is, to put it mildly, off-putting. In a conference talk, fine, but in a paper submitted to one of what I think are the two leading SLA journals (SSLA and LL), this is completely unacceptable.

So, I have to cite them all to be published in SSLA and LL?

Another parallelism; first one is from Journal Y and the second one is from Journal X

It is not clear from the introduction why you ask a question that focuses on the effect of the first language in your title? (in fact, if you read a bit more on EFL learners’ use of alternation phenomena, you’ll realize that no one is reporting that first language plays such an important role .)

First and as mentioned above, that is only true of one ignores literally dozens of studies by other scholars cited above and many others (see recent issues of the Interntl J of Learner Corpus Res, for example). But there is a second issue: Studies compared native and learner data because they addressed the bigger theoretical question ‘how do learners differ from native speakers?’. But, honestly, what even is the bigger theoretical question if one 1) only looks at learner data like here but 2) does not even include differences between learners from different L1s?

Despite my quotes from related studies (Whong-Barr and Schwartz (2002); Al-jadani (2018); Babanoğlu (2011); Kang (2011) and Song and Sung (2017); Jäschke’s (2016)) proposing L1 influence, the reviewer seems like ignored them all. Why ignore them and glorify the others?

Another one from the same reviewer;

But, honestly, what even is the bigger theoretical question if one 1) only looks at learner data like here but 2) does not even include differences between learners from different L1s?

How come it is a ‘bigger’ theoretical question? Who decides that?

Also figures from my study form comparing subcorpora i.e. learners from different first languages. Actually, there are 3 figures, from three different regression models.

Here is the reviewer’s respond to these figures;

‘this is what learners from completely different L1s are doing (and I didn’t bother checking for whether the learners from different L1s differ)’

Well, I did not know that all these languages are not different. Thanks reviewer first!

Another one;

It is sad irony that precisely the whole field of research this ms manages to ignore — Deshors, Ellis, Gries, Roemer, Szmrecsanyi, Wulff — has dealt with precisely these issues and has argued for precisely the methods that the present ms egregiously fails to consider/implement

This is what linguists call ‘priming’, Well it is priming at its best! Of course, reviewer also ‘ignored’ the point, where I cited a study (Chambaz and Desaguiller (2016)) proposing the flaws of the methods I ‘failed’ to implement. Why should I implement the method?

Another one;

1) Believe it or not, but this paper does not formulate explicit research questions! It just says it “follow[s] the framework provided by previous studies” (with the utterly incomplete coverage of the literature that this ms offers)

And here is my statement in the study,

Therefore, following the framework provided by previous studies, yet with a different methodology, this study analyzed English datives with a top-down approach, from constructions to verbs and other variables. Accordingly, the dative phenomenon was investigated not through a set of selected verbs but via a constructional perspective to assess the statistical difference in terms of overall dative construction among various learner corpora. In doing so, rather than judging learners’ output to native-speaker norms, corpora of learners with different L1 backgrounds were cross-compared to assess the degree of variation among each other and to reveal the common patterning of dative constructions in learner writings.

Because dear first reviewer, this is a partial replication of the original study, with a different framework.

Another one from the reviewer;

the paper says that, instead of the comparison to native speaker data, “corpora of learners with different L1 backgrounds were cross-compared [what’s cross-compared?] to assess the degree of variation among each other” yet it adopts a statistical design even the best of which explicitly makes that impossible.

I guess, the reviwer did not see (rather than failing to understand) figures where probabilities of observing such constructions in each subcorpus.

Another one from the same reviewer;

the third of the three statistical analyses does not include interactions of NativeLanguage with all other contextual predictors but instead uses NativeLanguage for random intercepts, which means that, AGAIN!, the intercept adjustments of NativeLanguage will just reflect the constructional frequencies given everything else in the model (but NOT how the L1 languages moderate the effect of the other predictors.

Do I really need to explain the concept of Random Effects? If so here it is;

In other words, the levels or groups in a random effect can be conceptualized as a sample of levels from a larger population of levels — some of which may not be represented in the model. An simple example of a random effect in a model might be the response of shrub height predicted by the categorical effect of forest type. More specifically, we might think that although our dataset only includes 4 different forest types in which we collected data — bottomland hardwood, longleaf pine, Aspen-Birch, and Redwood forest — that the four forest types we observed are just 4 samples from a larger collection or distribution of forest types out there.


So, we have 27 different data points/sources, each represeting a bigger group of population. If you say, this should not be ‘random effects’, then what?

Another from the same reviewer;

We are not told what the accuracy of spacy’s parsing is for the learner data, which is particularly relevant because the data are from different proficiency levels and, therefore, the parsing accuracy could not just be lower in general, but differently lower for different parts of the sample

Obviously you do not know how NLP works, sorry for that but at least you may have requested to be explained. How is the accuray of a statistical model relavent to learners’ proficiency levels? see details for Spacy here:

Anyways, there is soooo much to say yet I cannot write them all here. What I am trying to explain here is this: This is not what it means to be a reviewer. Journals do not ask anybody to be a reviewer so that they can glorify their own studies or the studies of their friends/students etc or to satifisy their ego. Unfortunately, this is how the academia in certain fields works. I also discuss and share ideas with a thousand of statistician or NLP experts, who are outside the academic circle who are so much more expert on these topics, yet, none so far has even tried to entertain their own selfishness or ego. Why academia?




Lecturer, Translation Studies, Second Language Learning/Teaching GünÇev platformu olarak öğrencilerimin yazı/makale/haber çevirilerini paylaşıyorum.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

School is Cool…kinda

Stories, Not Stats: About Diversity in Technology (Part 4 of 5)

Blog Post Feb. 2

Effects of Technology on Education

The Future of Work — The Students Aren’t Prepared

Respondus is the Perfect Reflection of In-Person Education一and that’s Exactly Why it’s So Terrible

Coursera Plus: Is the long term dedication worth it?

Are You Game? Understanding Student Stress and Making College Courses Fun

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Fatih Ünal Bozdağ

Fatih Ünal Bozdağ

Lecturer, Translation Studies, Second Language Learning/Teaching GünÇev platformu olarak öğrencilerimin yazı/makale/haber çevirilerini paylaşıyorum.

More from Medium

Empire State Trail -Cycle for Life

Lights out Carpathia

How to deal with Chaos and other things?

16 perspectives and practices to support ourselves through times of change