“Accurate Design of in vitro experiments – Why does it matter?” was the name of a previous post (LINK) from our August 2016 Newsletter, where we presented and discussed the concept of Pseudoreplication for in vitro experiments. Towards the end, we called for a meta-analysis of articles to get an idea about the ratio between biological and technical replicates.

Now, almost two years later, Stanley E. Lazic and his co-workers report the results of such an analysis, at least for in vivo experiments, published in PLoS Biol (LINK):
At the beginning, the authors make an important differentiation between 3 different entities, which are:
1. The Biological Unit (BU),
2. The Experimental Unit (EU),
3. The Observational Unit (OU).
It is important to be aware which one of these entities is actually the focus point of the investigation and will be used for replication. This will help to clearly distinguish between genuine replication and Pseudoreplication.

The purpose of an experiment is to test a hypothesis, estimate a property, or draw a conclusion about  a BU. Thus, the definition of the BU depends on the scientific hypothesis. If a study is designed to analyze whether the drug of interest will kill cervical cancer HeLa cells, it is sufficient to just use this cell line. However, if scientists aim to understand whether the same drug is effective against cervical cancer in general, it is necessary to test several more cancer cell lines and primary cell samples. In the first example, the BU is the HeLa cell line whereas in the second example, BUs are the multiple cervical cancers investigated.

The EU is the next important entity to be defined. It is usually the entity which is randomized or, in other words, the unit that receives a treatment allocation, e.g. person, animal, culture dish, or just one well in a microtitre plate. The BU and EU can be the same, but the sample size only corresponds to the number of EUs.

The OU is the unit of measurements, for example the length of dendrites, the body weight or the absorption of a sample. Increasing the number of OU does not increase sample size but will provide an idea about the technical error of an experiment.

Furthermore, three additional criteria must be met to achieve genuine replication, or, in other words, for a meaningful “N” that is not Pseudoreplication:
1. Randomization must be applied independently to groups, and if subgroups exist, randomization must be applied independently to groups within the subgroups,
2. Treatment must be applied independently and no spill-over must occur,
3. Samples should not influence each other.

If any of these three criteria is violated, samples cannot be treated as independent or EU anymore meaning that genuine replication is not provided.

Coming back to the meta-analyis, Lazic and his colleagues analyzed 200 papers which were chosen according to specific search criteria. Their analysis revealed that only 22% of articles contained experiments based on genuine replication. In contrast, 46% performed Pseudoreplication and about 32% did not provide enough information to make a judgement.

However, these numbers are derived from articles reporting animal research as these studies provide generally more information about sample size and randomization. One can most likely expect even further reduced numbers for in vitro experiments (due to a lower degree of awareness for the problem).

The authors conclude that “there are few ways to conduct an experiment well, but many ways to conduct it poorly. Without identifying the correct EU and having replication in the right place, experiments will likely be of little value or will test an uninteresting hypothesis.”