- Researchers identified a widespread reporting bias in medical reviews where benefits are found but not acknowledged in recommendations
- The bias was most visible in reviews of controversial interventions, including vaping for smoking cessation
- Authors often downplayed or dismissed positive findings using consistent narrative tactics
- The practice risks misleading clinicians, policymakers and the public about effective harm reduction options
A new scientific paper questions whether medical reviews can be trusted to tell the full story on health interventions like vaping.
Researchers found that even when reviews show clear benefits, authors often downplay the results or refuse to recommend the treatment. It’s a pattern that could mislead clinicians, policymakers and the public.
The paper, published in Research Integrity and Peer Review, examines dozens of systematic reviews and identifies a recurring pattern: review authors report statistically significant benefits in their results, but then fail to recommend the intervention – or even actively advise against it – in their conclusions.
The authors said: “..in many instances, systematic review authors reported findings favorable to the effectiveness of the treatment, yet they declined to recommend it or recommended against it despite the evidence of its effectiveness in their own systematic reviews.”
The findings have implications far beyond academic debate. Systematic reviews are often used to inform clinical guidelines, public health advice and regulatory decisions. When conclusions do not reflect the evidence presented, consumers and decision-makers may be left with a distorted picture of risk and benefit.
What is “reverse spin bias”?
The study introduces the term reverse spin bias to describe situations where authors narratively downplay or dismiss positive findings from their own data. The researchers define it as “the narrative discounting or dismissal of statistically significant findings of benefit.”
Traditionally, “spin bias” has been used to describe the exaggeration of weak or non-significant results. However, the authors argue that the opposite problem – minimising or ignoring statistically significant benefits- has largely escaped scrutiny.
In this analysis, reverse spin bias occurs when evidence shows an intervention works, but the final recommendation does not align with those findings.
Vaping reviews show the clearest pattern
The researchers began by examining systematic reviews of vapes for smoking cessation published between 2021 and 2025. Of 16 reviews analysed, most found that vaping was more effective than other quitting aids such as nicotine replacement therapy or no treatment at all.
Despite this, only a small minority of review teams recommended vapes as a cessation option. Several advised against their use, while others declined to make any recommendation – even when their own data showed clear benefit.
The study says that “thirteen systematic reviews reported findings that e-cigarettes were significantly more effective than other cessation methods but only three review author teams recommended their use as a cessation treatment.”
This disconnect between results and recommendations was not subtle. In multiple cases, statistically significant outcomes favouring vaping were followed by conclusions emphasising uncertainty, potential harms or a lack of confidence in the evidence base.
Not limited to vaping
To assess whether this pattern was unique to nicotine products, the researchers also examined recent systematic reviews of medical cannabis for pain management – another politically and socially contested intervention.
They found similar inconsistencies. Many reviews reported measurable pain relief associated with cannabis use, yet concluded that the treatment should not be recommended or that evidence was insufficient to support its use.
The consistency of these findings across two different interventions suggests the issue is not methodological error, but a broader narrative pattern in how controversial treatments are discussed.
Five common ways benefits are downplayed
Across the reviews analysed, the authors identified five recurring narrative strategies used to discount positive findings:
Discounting the evidence base
Reviews often described their own data as “limited” or “low quality” without formal justification, even when standard assessment tools were not applied.
Discrediting individual studies
Primary studies showing benefit were sometimes dismissed as weak or unreliable without reference to established risk-of-bias assessments.
Appealing to fear
Some conclusions raised concerns about unspecified future harms, even when adverse effects were not part of the review’s findings.
Rejecting the intervention on principle
In some cases, authors appeared to oppose the treatment modality itself – for example, nicotine use – regardless of comparative effectiveness.
Omitting favourable findings
A small number of reviews excluded or overlooked subgroup results that showed clear benefit.
Why this matters for harm reduction
The study warns that reverse spin bias may have real-world consequences. When effective interventions are framed negatively or dismissed, they may be excluded from clinical guidance and public health strategies. The authors note that the result is that “potentially effective treatments are neglected, and the data generated by clinical studies are wasted.”
For smokers who struggle to quit using conventional methods, this could mean reduced access to lower-risk alternatives, despite growing evidence that they can help people stop smoking.
The authors also raise concerns for research integrity more broadly, arguing that editorial and peer-review processes may be failing to challenge mismatches between evidence and recommendations.
A call for closer scrutiny
The study does not claim that vaping or other controversial interventions are risk-free. Instead, it calls for greater transparency and consistency in how evidence is interpreted and presented.
By naming and describing reverse spin bias, the authors hope to encourage editors, reviewers and readers to look more closely at whether conclusions genuinely reflect the data.
The paper concludes: “..editors and peer reviewers must be alert to discrepancies between the findings of biomedical systematic reviews and the treatment recommendations that their authors endorse.”
