All Case Study publications presented in this section have received a lot of interest among the scientific community and are particularly interesting from a GSP perspective. These case studies demonstrate the need for a high-quality study design, including the incorporation of basic GSP standards and guidelines, which is the foundation of all PAASP services provided.

Randomization reduces bias by equalizing factors that have not been explicitly accounted for in the experimental design. Most scientists frequently use the concept of randomization when allocating the experimental units across the treatment groups.

However, there are two additional, less well appreciated aspects where randomization can crucially improve the study design of an experiment: The first is the spatial arrangement of samples (spatial effects, e.g. when dealing with multi-well microtiter plates with edge-effect artifacts) and the second are observations correlated in time (order effects). A typical example of such a time-dependence between sample number and the main outcome variable are the measurements of stress hormones with a circadian rhythm, where it can be difficult to distinguish treatment effects from circadian effects without appropriate randomization.

A striking example for order effects, where the lack of randomization can lead to erroneous conclusions, is discussed below in the context of a homeopathy experiment – a remarkable account of what happened when a magician offers $1 million for a positive result:

Professor Madeleine Ennis of Queen’s University, Belfast was unimpressed by claims that water had a memory and that effects of homeopathy can be measured – till she tried the dilution experiments for herself. When she did so, she was astonished to find that she too obtained results showing that basophil degranulation is regulated by histamine through a negative feedback process even at homeopathic concentrations (Brown and Ennis, 2001) http://link.springer.com/article/10.1007/PL00022402.

In 2002, a team of the UK’s BBC2 science series Horizon, including the Vice-President of the Royal Society, Professor John Enderby, the statistician Professor Martin Bland and the magician James Randi, challenged her results by conducting their own scientific experiment (Bland 2005) abstract. To make it even more dramatic, J. Randi offered a prize of $1 million to anyone ‘able to demonstrate a supernatural ability under scientific testing criteria’, which would cover homeopathy as well.

The idea of the Horizon experiment was to make homeopathic dilutions of histamine and, as a control, of pure water diluted and agitated in exactly the same way. After the blinding process, each sample would then be divided into two parts and sent to two different laboratories. In each laboratory an assay would be carried out where the contents of the test tube would be added to fresh blood and the number of activated and inactivated basophil cells would be analyzed via flow cytometry measurements. Each dilution would be used for blood from five different donors in each laboratory.

When M. Bland received the data and prepared the analysis for the day of the code-breaking, he arbitrarily compared the first half of the observations with the second half obtained in Laboratory 1 (left Figure). To his amazement, he found a significant difference between these two sets of data. When plotted the mean percentage of active cells across the five donors (percentage active) against the sample number, that is, the order in which the measurements had been made (right Figure), he found a strong cubic relationship.

Left Figure: Percentage active basophils by order in which samples were measured (Lab 1 only)
Right Figure: Percentage active basophils against sample number (Lab1 only)

Did indeed the first half of test tubes measured contain the histamine dilutions, which had inhibited basophil activation and the second half contain the water controls? If this were to be the case, James Randi had lost $1 million and at the same time, the result would force scientists to rethink some of their fundamental beliefs – and water suddenly got a memory.

Importantly, however, the Horizon experiment was rigorously blinded – in such a way, that the order in which the dilution test tubes were presented and used for each blood sample was strictly randomized, thereby preventing order effects from producing spurious results.

It was found later that the measurements based on living blood cells vary greatly over time as the blood itself changes over the course of the measurement session. There is thus the possibility of a relationship between the measurement and the time at which it was made. If researchers were to randomize the order in which measurements were made, this time effect would form part of the random error rather than produce a spurious treatment difference.

In the paper of M. Ennis, however, no mention of blinding or randomization was found.