Ilya Lipkovich, Eli Lilly and Company, and Alex Dmitrienko, Mediana Inc.

Recently, the Biopharmaceutical Section conducted a survey of the editors and associate editors of several applied statistics journals with a biopharmaceutical orientation (Statistics in Medicine, Statistics in Biopharmaceutical Research, Journal of Biopharmaceutical Statistics, and Pharmaceutical Statistics).

The survey was designed to be open-ended and asked the editors to list—based on their experiences—three don’ts commonly seen in submitted manuscripts and three dos. The editors were encouraged to avoid listing well-understood don’ts such as plagiarism and focus on less obvious dos that would be appreciated by statisticians preparing manuscripts.

We received 27 responses, each consisting of multiple dos and don’ts. Most respondents provided three of each kind, though some provided mostly don’ts or dos. Because our sample size fell short of n=30, we refrain from any formal statistical inference.

Although the immediate motivation for conducting this survey was the desire to understand why so many of our own manuscripts have been rejected by these journals, we think the responses will be useful to all authors of statistical manuscripts, whether novice or seasoned.

For convenience, we divided the areas covered by the respondents into six broad categories, presented below with detailed suggestions. As both authors serve (or have served) as associate editors for Statistics in Medicine, we felt free to add our own suggestions here and there.

Context of Research and Motivation

These include the need to provide proper motivation for research with such critical elements as alignment with existing literature, novelty, and applicability.

• DO include a convincing and easy-to-follow real-world example illustrating the problem and motivating your research.
• DO conduct a thorough and unbiased literature review ensuring good alignment with existing research. This not only means the paper should present novel approach(es), but it also should avoid inventing new terminology and introducing new notation when established ones exist. As in almost any area of human endeavor, a good strategy is to stick to existing standards unless you can propose better ones.
• DON’T include excessively long literature review, unless you are writing a review paper. Only cover literature relevant for your research.
• DON’T try to solve artificial or nonexisting problems. Always ask yourself, “Is the method proposed applicable to a real-life problem?” Artificiality comes with many faces. For example, do not do research just to provide a Bayesian counterpart to a problem where an existing (frequentist) solution works well. Often authors “invent” methods involving multiple steps that somewhat arbitrarily combine existing procedures with little insight into why they should work better than available methods.
• DON’T try to solve a special case when a more general problem has been solved already.
• DON’T try to publish two very similar papers with a different order of authors.
• DON’T write in the introduction that “unfortunately no approaches exist to handle this problem”; it is, in fact, quite fortunate for your research.

Structure and Style of Presentation

The key attributes are the length, logical structure, efficient use of tables and figures, use of appendices, and supplemental materials.

• DON’T write long manuscripts (stressed by one-quarter of responders).
• DO make sure the structure of the paper is well thought out. As our respondents did not provide examples of poor structure, we would like to make a couple specific suggestions:
  • Although clinical journal standards require you to not disclose results before the “results” section, keeping the reader in suspense, this pedantic rule is not followed by many influential statisticians of our time who often present the summary of key findings in the introduction.
  • Do not write a history of your research, tracking how the ideas evolved in the course of writing your article. Present the final view.
• DO present information efficiently. Whenever possible, graphical summaries are preferred over tabular summaries (in addition, tables can be moved to the appendix).
• DON’T have an excessive number of tables and figures in the main text.
• DO provide detailed annotations for each figure that allow the reader to understand the graph (and the context), even without reading the description in the text. This is also helpful for automatic generation of article summaries, as machine learning algorithms “like” to have figures explained by surrounding text.
• DO present proofs and other highly technical details in appendixes.
• DO make sure the paper is proofread by a native English speaker.

Simulation Design

• DO make sure simulations cover relevant cases.
• DO explain in nontechnical language why you chose particular scenarios.
• DON’T choose only the scenarios that favor your method. Show how your method performs when the assumptions are not met.
• DON’T choose as a comparator for your method a “strawman” (a pseudo-standard that is easy to beat). Compare your method to a broad class of alternative approaches. In most settings, there is no method that is uniformly better than all others, and it is important to identify the cases in which alternative methods are superior to your method.
• DON’T use simulations when you can make a point using an analytical argument.
• DON’T present results with nine decimal places, especially if you have run only 100 simulations per scenario.

Balancing Theory with Applications

• DON’T overload the paper submitted to an applied statistics journal with mathematical equations and statistical jargon. The main results should be explained using language understandable by an intelligent nonmathematically-oriented reader.
• DO provide intuition behind mathematical results.
• DO provide a real-life example (see also Context of Research and Motivation).

Properly Framing Your Contribution

• DON’T “oversell” and exaggerate the importance and novelty of your paper.
• DON’T write your paper like a promotional dossier explaining how everyone is doing things wrong and “here we come and solve all the world’s problems.”
• DO provide a critical account of your research, stating gaps and limitations. Stating limitations of the proposed method is important; however, admitting your sins does not automatically mean forgiveness.

Reproducibility

• DO describe methods in sufficient detail that can be reproduced.
• DO make the code and data sets available.

We would like to conclude this article with a DO (or rather a BE) suggested by one of our responders: “Be Brilliant!”