False Positive .Be Aware of it
There has been an increased concern over the science of psychology and how the research is conducted. This concern was increased after the replication crisis; replication is the process of repeating previous research and reaching the same conclusions and results, replication is very important to the validity of science and in preventing false positives; were 9 out of 14 replication attempts conducted by Nosek and Lakens (2014) failed to replicate, and the other 5 were partially replicated. These failures put psychology under scrutiny.
Our job as scientists is to understand the world around us. We make assumptions and hypothesis about events, collect data, and analyze it to accept or rejects those hypothesis, but errors are inevitable. One of the costly errors we make is the incorrect rejection of the null hypothesis or false positives (Type 1 error). Prestigious journal can make a mistake by publishing papers that have significant results or too good to be true results and later is discredit it due to the type 1 error, an important examples, the cold fusion claims by Fleischmann and Pons (1989) and the correct prediction of events in the future, Bem (2011). Both experiments failed to replicate despite many attempts to validate their results.
In an important article, Simmon et.al (2011) noted how “undisclosed flexibility in data collection and analysis allows presenting anything as significant.” In their article they focused on P-hacking by topping up subject until significant results is reached, increasing the risk of false positive (Type I error). Several recently published articles(Bakker & Wicherts, 2011; Fiedler, 2011; Garcia-Perez, 2012) have highlighted that research practices in psychology have the tendency to be influenced by many factors (median split, sample size, etc.) that will increase false positive rates.
Scientists, weather intentionally or unintentionally, might take advantage of factors that will influence false positive rate. The problem of false positive cannot be overemphasized, as to it will mislead our understanding of human behavior and waste vast amount of resources. However, the need for guidelines or recognition of factors that will reduce false positive rate (type1error).
Kou Murayama, Reinhard Pekrun, Klaus Fiedler (2013) discussed three factors that exist in current research parties that will help reduce type 1 error. The first is theoretical prediction, as the empirical evidence accumulate in support of the theory in turn forcing researchers to conduct conservative tests which reduce type1 error. The second factor is including multiple studies in the same paper, additional studies might address problems in the first study and thus providing a stronger evidence for their hypothesis. The third factor is sequential data collection (data collection), the additional information should be used for the observed p value, not just to thoughtlessly gather additional data to support the hypothesis. Researchers should be careful when conduction research and reporting data to prevent false positive. A full balanced view of factors that increase false positive and factors that reduce false positive should be considered when conducting research. Such consideration will help reach a more meaningful conclusions, better report p-values, and help better understand the world we live in.
Bakker M., Wicherts J. M. (2011). The (mis)reporting of statistical results in psychology journals. Behavior Research Methods, 43, 666–678. doi:10.3758/s13428–011–0089–5
Bem, D. J. (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100, 407–425. http://dx.doi.org/10.1037/a0021524
Fiedler K. (2011). Voodoo correlations are everywhere-Not only in neuroscience. Perspectives on Psychological Science, 6, 163–171. doi:10.1177/1745691611400237
Garcia-Perez M. A. (2012). Statistical conclusion validity: Some common threats and simple remedies. Frontiers in Psychology, 3, 325. doi:10.3389/fpsyg.2012.00325
Murayama K.,Pekrun R.,Fiedler K.(2013). Research Practices That Can Prevent an Inflation of False-Positive Rates. Personality and Social Psychology Review, Vol 18, Issue 2, pp. 107 – 118Nosek, B. A., & Lakens, D. (2014). A method to increase the credibility of published results. Social Psychology, 45, 137–141. http://dx.doi.org/10.1027/1864-9335/a000192
on Psychological Science, 9, 40–48. http://dx.doi.org/10.1177/1745691613513470
Simmons J. P., Nelson L. D., Simonsohn U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366. doi:10.1177/0956797611417632
