We like to cite Dorothy Bishop who wrote several years ago: “We know how to formulate and test hypotheses in controlled experiments. We can account for unwanted variation with statistical techniques. We appreciate the need to replicate observations. Yet many researchers persist in working in a way almost guaranteed not to deliver meaningful results (REF)”.

Why do we fail to change? Or, why do we not change faster?  We guess the simple answer is that there is still not enough evidence that good research practices means greater success and, vice versa, that not following the recommended practices is indeed a recipe for failure.

Archimedes is quoted saying “Give me a lever long enough and a fulcrum on which to place it, and I shall move the world.”  So, we need a fulcrum, i.e. some solid evidence, to change the currently dominating practices in research.

In this context, we see it as one of our tasks to identify and share arguments that would convince scientists to start changing now.  Obviously, such arguments need to be strong and need to appeal to individual scientists.  For example, EQIPD has released a set of examples for “Why Quality Matters” for different roles in academic and industry research from students to principal investigators and quality professionals.

We have also built collaborations with experts in animal care and use to develop arguments based on the ethical concerns of not designing and conducting robust studies.  We have also discussed potential negative consequences of low quality data sets regarding intellectual property rights and patentability (LINK; see also here LINK).

And it looks like our “family of arguments” has a new member:

Like many of our readers, we have been following the story of Cassava Sciences since it started to develop last summer when a citizen petition to the U.S. Food and Drug Administration (FDA) was filed raising concerns over the quality of studies Cassava conducted about its lead drug simufilam and asking the FDA to halt an on-going late-stage study (LINK). By the time the petition was filed, Cassava had already released clinical data that looked positive.  Thus, the key question is: Can a drug with promising clinical results, a strong board of scientific advisors, and so much hope for Alzheimer’s patients could possibly fall apart because of faulty experiments with allegedly manipulated results at its core?

In February this year, the petition was denied on procedural grounds, without addressing the various accusations (LINK).  

However, it should be noted that there is no smoke without fire (https://www.cassavafraud.com):  Evidence summarized in the original petition was not countered by the company, papers were retracted or flagged with “expressions of concern”.  And, in a recent article in the New York Times, Thomas Südhof of Stanford labeled Cassava’s story as “implausible and contrived” (LINK).

This is not the first time when questionable practices in preclinical research cause biotech some trouble with Athira Pharma as one recent example (LINK). 

However, the Cassava case is unique.  This is the first time, to our knowledge, when deviations from best practices or issues with data integrity are actually used by someone with the intent to benefit from it – either being associated with Cassava’s competitors (as Cassava’s management has suggested) or by holding a short position in Cassava’s stock (a short position allows an investor to profit from a drop in the company’s stock price).

Why do we find this so special? If successful, this could motivate others to look for such deficiencies and try to use the identified evidence (i.e. lack of data quality) in a similar way.  It would then be a clear signal to biotech to stick to better research practices from the very beginning of their projects.

Will it go that far? 

Cassava’s Phase 3 studies are currently running.  If these studies fail, we will have certainly some more arguments when discussing the negative consequences of deviating from good research practices but still need to ask ourselves why we are not able to stop such projects from raising hundreds of millions in dollars/euros – and thereby depriving better deserved projects from getting funded. 

However, if these studies should succeed (despite the data quality issues), all our arguments are gone since the winner is always right…