Search
Search Menu

When p-hacking is not p-hacking – or: How shall the data be reported?

Organ bath experiments are a key technology to assess contractility of smooth muscle. When comparing force of contraction, investigators typically normalize measured force for size of the tissue specimen as a larger piece of tissue should develop greater force. However, there is a lack of agreement which parameter should be used as denominator for normalization.
 
To analyze the effects of ageing on muscarinic receptor subtypes and function in rat urinary bladder, Tim Schneider and colleagues pre-defined tissue specimen length as denominator in their study protocol to normalize measured force of contraction. Using this as the primary outcome parameter, the investigators showed that carbachol, a synthetic mimetic for the neurotransmitter acetylcholine acting primarily by stimulating muscarinic receptors, contracted bladders from young and old rats with similar efficacy and no difference could be detected between the two groups (old vs young).
 
Given the uncertainty around the best normalization approach, the investigators also explored normalization for strip weight (as secondary outcome parameter) and found that maximum force of contraction was significantly smaller in old than in young rats. It became clear that whether carbachol effects differed between groups depended on the type of data normalization.
 
Having obtained the different data sets based on the pre-defined parameters in the study protocol, the question now arose how best to report the data obtained?
Here, the investigators aimed for maximum transparency and decided not to select one analysis over the other but to report both negative and positive analysis outcomes in the same publication. In addition, they clearly stated which parameters were defined in the study protocol as primary and secondary outcomes.
Only several years later could a much larger study by the same group show that normalization for tissue specimen weight would have been the right choice and more informative.
 
This is an example that openness and transparency are a key basis for trustworthy research and that there should always be sufficient room in scientific articles to report results as detailed as necessary for the reader to understand the complexity of a study.

Share this

Leave a Comment

Required fields are marked *.


This site uses Akismet to reduce spam. Learn how your comment data is processed.

Scroll Up