Books that can be summarized in three sentences (https://jamesclear.com/book-summaries). And one can summarize in a couple of sentences what most movies are about. We frequently produce such short summaries to provide a quick overview about the format and content of such media.
Like many of our colleagues, we have read the recent analysis that appeared in PLoS Biology with great interest. In this report, Wieschowsky and colleagues analyzed a sample of 109 investigator brochures (IBs) presented for ethics review at 3 institutional review boards based at German Medical Faculties between the years 2010–2016. For 708 unique preclinical efficacy studies (PCESs) described in these IBs, the authors find that “less than 5% of all PCESs described elements essential for reducing validity threats such as randomization, sample size calculation, and blinded outcome assessment.”
Technically, this analysis is well conducted and the results are presented very clearly. However, Wieschowsky et al. do not explain the relationship between IBs and original R&D reports that serve as a basis for such IBs. To make it clear – the relationship between IBs and original reports can be compared to a) either watching a movie or b) someone telling you in very few sentences what the movie is about. The original reports can be up to 1,000 pages or longer (particularly for preclinical safety studies conducted under GLP standards) and typically contain all details one would expect in a conventional scientific publication (including elements essential for reducing biased research outcomes such as randomization, sample size calculation, and blinding procedures). In contrast, IBs include essentially only short abstracts of these reports (occasionally with the most critical data sets summarized in a table or graph format) and are indeed not intended to provide all experimental details regarding study design, conduct and analysis.
Unfortunately, the existence of such reports and their role in regulatory review and approval is not discussed and mentioned in the PLOS Biology paper. Without this information, and as a consequence, the conclusions as they appear in paper by Wieschowsky et al. could and actually are ill-received by the general public that sees in this analysis yet another example of how Pharma Companies are misleading doctors and patients and how the evidence on which new drugs and products are based is systematically distorted by the pharmaceutical industry. To represent the most complete and comprehensible picture of the drug development and regulatory review process, it is therefore essential that pharma companies, regulators and the Institutional Review Boards (IRBs) themselves will explain the documents submitted for regulatory reviews (such as IND and associated reports) and the purpose of the IBs.
Importantly, however, the last paragraph of the above discussed PLOS Biology paper does address a critical issue that highlights the differences in presenting preclinical efficacy and safety studies. For the latter, it is generally sufficient to state that a study was done under GLP standards and according to a corresponding OECD guiding document. It is therefore possible to simply focus on describing the most essential results without omitting important information. In contrast, for efficacy studies, there are no guidelines or commonly accepted and followed procedures that could be referenced to in order to indicate that certain standards have been met.
It could therefore be beneficial for all interest groups that regulators address this issue and start developing standards for the design and reporting of PCESs that would facilitate unambiguous reporting and summarizing data sets in IBs.