What makes a good applied methods paper

By Dimitris Rizopoulos and Jeff Leek

We have been co-editors at Biostatistics for about three months now. Over that time we have started to observe some trends in the types of papers we get most excited about and are most interested in sending to review. We thought it would be helpful for authors both at our journal and other methods journal to get an idea for what we are looking for as editors.

While these are the important characteristics of methods papers, we are also open to analysis papers, re-analysis papers, new applications of old methods, software papers, and commentary on current science issues from a statistics perspective.

We think these characteristics apply across a broad range of statistical and computational methods development and they certainly make us more interested in a paper if they are true. Not every good paper has every characteristic below, but as a general rule a good methods paper

1. States the scientific problem the method is trying to solve at the beginning of the abstract and introduction.
2. Describes a recent and relevant data set where this scientific problem occurs.
3. Describes a method, justifying decisions based on how they are connected to the scientific problem
4. Performs a thorough comparison to the best methods available through simulation or data examples. We don’t expect authors to implement other people’s methods in software just to compare them, but we would like to see some comparison to the best methods that have software available.
5. Explains the metrics for comparison and why the comparison shows the proposed method represents a practical advance.
6. Applies the method to the example data set and explains the solution to the problem.
7. Includes well-documented scripts and data that perform the analyses presented in the paper.

Methods papers that have all of these characteristics are the most likely to have a major impact on the field. We also think that methods papers that provide a link to a software library or package that implements the method — with complete documentation and ideally on a curated repository like CRAN or Bioconductor — can raise the impact of a paper substantially.

These are the characteristics of papers we are looking for at Biostatistics because we believe #methodsmatter.