We’ve all had the problem arise where you try to repeat work that has been published in the literature just to find that it seemingly isn’t reproducible. Sometimes, it’s just a matter of tweaking the prep a little to make it suit you better, but others, no matter what you do, just can’t be repeated. What is there to be done about this?
Well, in the field of psychology, they are currently undergoing what they call a ‘irreproducibility crisis’, whereby confidence in the field is being lost by the findings of many studies being unable to be repeated. This article, published on the Nature website, highlights suggestions by experts which suggest that PhD students should be required to reproduce work from the literature before being allowed to graduate.
It’s certainly an interesting prospect. Although the article focuses on the field of psychology, it does bring about the idea of requiring chemistry and related PhD students to successfully repeat a published synthetic procedure or analytical study before they are deemed competent enough to be able to graduate.
In a way, I can see the merit in this idea. If a reliable set of published work can be found, it could be a useful way of testing a chemist’s practical ability before they’re able to gain their PhD and go out into the world of research. However, how can we be sure that the problem of irreproducibility may be down to an unskilled researcher, and not just the unreliable work that has found its way into the literature?
Furthermore, comments in the article suggest that the task of reproducing work should go to more senior researchers who have the expertise to carry out research with a higher level of skill, and who have the experience and background knowledge to know why it may not be working. Also, experts are wary that forcing students to carry out this work will make it appear as “grunt work” and just be taking advantage of the cheap labour offered by graduate students.
There is also a suggestion that reproducibility work should be more widely published, so as to encourage researchers to carry it out, and should be more highly regarded on CVs and by hiring committees. If this was the case, perhaps it would pave the way for researchers to “clean up the literature“.
This is definitely an interesting topic, and one which will certainly divide opinion. What do you think? Have you had experience with poor reproducibility in the literature?