The “Lessons Learned” category: When negative results are useful

2018-07-23

If you’ve been doing science for more than a week, you know that failure is part of the process. But I would say there are two kinds of mistakes, roughly speaking: those we should have known about ahead of time and those we could not have.

The LiveCoMS Lessons Learned category is more-or-less for the “could not have known” category. In particular, a Lessons Learned article should be useful to a significant number of readers, teaching them about problems they would be likely to encounter in the absence of your paper. The category is intended to reinforce the overall journal goal of improving the quality of molecular simulation studies, in part by incentivizing dissemination of important insights into the nature of the simulation methods or systems, revealed through failure, that otherwise would not be published.

Let’s be a little more concrete. You calculate a potential-of-mean-force for your favorite ion channel using a well-established protocol with more sampling than has been used in the past … but there are clear signs the results aren’t converged. I think that’s valuable information to have in the literature. If it just goes into your ‘unpublishable/’ directory, then you are dooming your colleagues to repeat your mistake. But if you carefully document the problems and then go further to show it’s not only an issue for your pet system but may be a problem more generally, then you really have made an important contribution to science.

Another example. You perform a massive and systematic molecular dynamics study and show convincingly that your sampling is reliable using statistical methods … but your results clearly conflict with experimental data. Waste of time? Maybe you think so, but if you keep it to yourself and the system is really an important one, then we can expect others to repeat your mistake. Instead, bring this to the attention of the community and especially force field developers.

How do Lessons Learned fit into the “living” journal model? First, as with any of our articles, they can be updated by the original authors as needed. Second, our vision for these articles is that each will cover a category. For example, one article might be entitled “Lessons Learned in PMF calculation for transmembrane systems.” As such, Lessons Learned articles would effectively be companion pieces to the Best Practices articles. In the transmembrane case, the article would be initially be published by authors that already have a significant body of data on a system in the category. Additional authors studying other transmembrane systems could then add to the article, gaining authorship. All this happens via the process laid out in our author instructions.

My strong opinion is that our field will not benefit in the long run from suppressing negative results (as happens now implicitly by self-censorship) … or worse still, by pretending data is OK when it’s not.

Join me in helping our community to do its best. Submit!

–Dan Zuckerman Lead Editor, Lessons Learned Section Living Journal of Molecular Science