In 2012, Glenn Begly was serving as director of haematology and oncology at the pharmaceutical company Amgen. As part of his job, he monitored the development of new cancer drugs and he wondered why so many of the projects that at first glance seemed hopeful were failing. He found that most of them failed at the preclinical stage: they were unable to reproduce the published results. Begly kept investigating and in his article “Raise standards for preclinical cancer research”, he argued that only 11% of cancer studies could be replicated. Since then, the reproducibility problem has been made evident in other disciplines. A 2016 Nature survey revealed that, in the field of biology, over 70% of researchers were unable to reproduce the findings of other scientists and approximately 60% of researchers could not reproduce their own findings.
The lack of reproducibility in scientific research has negative impacts on health, slows down scientific progress, negatively affects the public’s trust in science and wastes time and money. In fact, a 2015 meta-analysis of past studies revealed that failed attempts to replicate preclinical work in biomedicine cost $ 28 billion per year in the United States alone.
Unfortunately, the field of synthetic biology is no exception. The sensitivity of biological systems to small changes in their cellular or environmental context makes it difficult to reproduce or build on prior results in the lab and to predict desirable behaviours in a deployed application. However, the engineering approaches applied in synthetic biology often demand quantitative precision in models and measurements. That is why synbio must tackle current inconsistencies in measurements and devise new methods to enhance reproducibility of experiments. [1]
Overly honest methods
Failures of reproducibility cannot be traced to a single cause, but there are different agents contributing directly to this issue. The truth is that people doing science have to work with all kinds of constraints: how worn their equipment is, how frequently lab personnel can tend to cell cultures, how precisely pumping rates can be controlled… In 2013, scientists started to share methodology jokes on Twitter reacting against the stuffy and sometimes unclear way that scientific papers are written and, due to the limitations scientists have to deal with, there was more than a grain of truth in most of the tweets:
«Incubation lasted three days because this is how long the undergrad forgot the experiment in the fridge,” read one tweet with the hashtag #overlyhonestmethods. “This dye was selected because the bottle was within reach,” said another. And more users took to the hashtag: “PCR reaction repeated for 25 cycles because that’s how long it takes to go to teach class.” Or: “Experimental time points where chosen so I didn’t have to come into the lab in the middle of the nigh.” Relatable content!
For scientists to be able to reproduce published work, they must be able to access the original data, protocols, and key research materials. Experimental protocols are often specific to a laboratory or even to a single researcher. So, one way to increase reproducibility is to improve reporting of data and protocols. Data repositories and online protocol editors, in which scientists can share and edit protocol documents, accelerate progress and reduce redundant efforts. In addition, these platforms can help to improve the standardisation of common techniques, improving experimental reproducibility.
One example is Protocols.io, and open access platform for detailing, sharing, and discussing molecular and computational methods that can be used at any point in the research cycle. This not only allows other researchers to faithfully reproduce the methods of another, but it also provides a paper trail of any method allowing scientists to see the evolution of the protocol over time. [2]
Automation and standardisation
Many of the techniques used within synthetic biology are performed on a routine basis, which makes them the perfect target for automation. However, the hight cost of automation technology has hindered its adoption by researchers.
Fortunately, some efforts to facilitate access to these technologies are already underway. One example is Opertrons, a company that makes robots for biologists to automate experiments that would otherwise be done by hand. They offer affordable products, ease of use and are open to customization. Automation will improve the reproducibility of experiments by eliminating human variations, and will also increase their throughput, allowing the generation of a much larger amount of data. [2]
Synthetic biology requires standards to cope with the hight degree of uncertainty in biological systems. Standardisation of biological parts will help to properly address the enormous cellular complexity of biological systems. In addition, by agreeing on measurement, units, tools, protocols, biological parts, devices and organisms, experimental results will be more comparable and reproducible, despite being obtained in different locations by different researchers. [3]
Synthetic biology has the potential to revolutionize sectors such as healthcare, drug production, energy sourcing, waste treatment, agriculture and much more! However, the development of this promising field won’t properly take off if it does not move towards engineering principles of predictability and reproducibility. Setting standards may be the first step for achieving these goals.
References
[1] Beal J, Haddock-Angelli T, Gershater M, de Mora K, Lizarazo M, Hollenhorst J, et al. (2016) Correction: Reproducibility of Fluorescent Expression from Engineered Biological Constructs in E. coli. PLoS ONE 11(6): e0157255.
[2] Jessop-Fabre Mathew M, S. N. (2019). Improving Reproducibility in Synthetic Biology. Frontiers in Bioengineering and Biotechnology, 7, 18.
[3] Tas H, Amara A, Cueva M.E, Bongaerts E, et al. (2020) Are synthetic biology standards applicable in everyday research practice? Microbial Biotechnology 13( 5), 1304– 1308