The resistance among the older generation that highlights remains highly problematic. It has at times affected my job satisfaction, my mental health, and my career trajectory. If you are senior, please express your support openly. It really helps
Nature: drops truth-bombs on irreproducibility and offers an optimistic prognostication that the scientific community is on the path to solving it.
1. Publication bias ❌ 2. Low statistical power ❌ 3. P-value hacking ❌ 4. HARKing (hypothesizing after results are known). ❌
1. Publication bias ❌ 2. Low statistical power ❌ 3. P-value hacking ❌ 4. HARKing (hypothesizing after results are known). ❌
I think that, in two decades, we will look back on the past 60 years — particularly in biomedical science — and marvel at how much time and money has been wasted on flawed research.
. urges researchers to rein in the four horsemen of the reproducibility apocalypse: publication bias, low statistical power, P-value hacking and HARKing.
1. Publication bias ❌ 2. Low statistical power ❌ 3. P-value hacking ❌ 4. HARKing (hypothesizing after results are known). ❌
Rein in the four horsemen of irreproducibility (). I would also add incomplete reporting, which is a huge (easily fixable?) problem. If we don't know what has been done and how, then we can't reproduce anything.
Scientific journals are clogged with false findings, for four main reasons: -Publication bias -Low statistical power -P-value hacking -HARKing (hypothesizing after results are known) ...but we're finally making progress solving the problem! By
The four horsemen of the reproducibility apocalypse: publication bias, low statistical power,  p-value hacking, and HARKing (hypothesizing after results are known)
"I think that, in two decades, we will look back on the past 60 years — particularly in biomedical science — and marvel at how much time and money has been wasted on flawed research."
There are 4 primary questionable research practices that undermine science (the "4 horsemen of irreproducability*"). They are all important, but I'm curious which people think is most important. Which leads us most astray? *see 's recent column
1. Publication bias ❌ 2. Low statistical power ❌ 3. P-value hacking ❌ 4. HARKing (hypothesizing after results are known). ❌
Nice to learn how stumbled on p-hacking, long before it joined HARKing, publication bias and low power, the 4 horsemen of irreproducibility. NIce also to see that social media are one of the powers to slay them.
Dorothy Bishop: Rein in the four horsemen of irreproducibility(Rein in the four horsemen of irreproducibility: publication bias, low statistical power, P-value hacking and HARKing (hypothesizing after results are known)
Excellent short piece on (ir)reproducibility by .
Dorothy Bishop: Rein in the four horsemen of irreproducibility(Rein in the four horsemen of irreproducibility: publication bias, low statistical power, P-value hacking and HARKing (hypothesizing after results are known)
ICYMI: 'Rein in the four horsemen of irreproducibility - Dorothy Bishop [, ] describes how threats to reproducibility, recognized but unaddressed for decades, might finally be brought under control' via #OpenScience
In #epigenetics, the fifth horseman is what I call the ghost hypothesis. Framing the study with a meaningless statement like “we looked for the epigenetic mechanism”. Any result then works. Rein in the four horsemen of irreproducibility
. urges researchers to rein in the four horsemen of the reproducibility apocalypse: publication bias, low statistical power, P-value hacking and HARKing.
Now that’s what I call a strong opening. Great piece by on tackling the problems of irreproducibility in research -
This. So much this. Rein in the four horsemen of irreproducibility
As the Four Horsemen of the Reproducibility Apocalypse make an appearance in , this message of redemption from is timely & important, across multiple fields of science:
A note on the reproducibility 'apocalypse' -- "The problems are older than most junior faculty members, but new forces are reining in these four horsemen."
“many researchers persist in working in a way almost guaranteed not to deliver meaningful results. They ride with...the four horsemen of the reproducibility apocalypse: publication bias, low statistical power, P-value hacking and HARKing“
Strong editorial comment from Rein in the four horsemen of irreproducibility
Tomorrow I organize a meeting on pre-registration with talks by & on writing pre-analysis plans. This piece in Nature today emphasizes the importance and the change that is happening
It is promotion time. A pleasure to write letters for colleagues. Criteria? Has anyone seen a university promotion document that specifies ‘candidate should provide evidence of a sustained track record of highly reproducible research’ Did not think so....
Just as I started to get papers on misuse of statistics and P-values for a #Collection in look what I found: Bishop on "four horsemen of the #reproducibility apocalypse: publication bias, low statistical power, P-value hacking and HARKing"
"The four horsemen of the reproducibility apocalypse: publication bias, low statistical power, P-value hacking and HARKing (hypothesizing after results are known)."
Important read: "Rein in the four horsemen of irreproducibility" (by )
Dorothy Bishop on how to rein in the four horsemen of the reproducibility apocalypse: publication bias, low statistical power, P-value hacking, and HARKing (hypothesizing after results are known)
Thank you ⁦⁩! These ideas are important for our field. The new options (eg ⁦⁩) and structures (eg ⁦⁩) have allowed changes I could not have imagined possible only a few years ago.
Rein in the four horsemen of irreproducibility
Rein in the four horsemen of irreproducibility
Four horsemen of irreproducibility: Publication bias - I have my personal (effective maybe!) combat on this field, Low statistical power, P-hacking, HARKing
irreproducibility in research and publication bias, translates to waste of money, resources and valuable scientific careers. Maybe one day...
This opinion piece is as relevant as ever. I disagree that we are doing better now. Rein in the four horsemen of irreproducibility
Rein in the four horsemen of irreproducibility
Rein in the four horsemen of irreproducibility
Remind me to explain to all you hypothesis testers about modeling/computational research, where no p-values are computed, no exploratory vs. confirmatory dilemma exists, and it’s unclear if publication bias applies #reproducibility
Rein in the four horsemen of irreproducibility
まさに「意味のある結果が出ないことが保証されている研究のやり方を多くの研究者が使い続けている」 「役に立つかどうかは重要ではない」とか「どんな研究も長い目で見れば役に立つ」とかいう議論に私が与しない理由はこれ
A different way of thinking about meta-analysis. It isn’t broken, it is doing exactly what it’s supposed to and very well: detecting systematic effects. It’s just that the systematic effect it’s detecting is pub bias. From
Rein in the four horsemen of irreproducibility
“the four horsemen of the reproducibility apocalypse: publication bias, low statistical power, P-value hacking and HARKing (hypothesizing after results are known)”. A few others as well.
Rein in the four horsemen of irreproducibility
Rein in the four horsemen of irreproducibility "The four horsemen of the reproducibility apocalypse: publication bias, low statistical power, P-value hacking and HARKing" 🙏🙏 If you do not know (one of) them, please read this. 🙏🙏
Rein in the four horsemen of irreproducibility: Prof describes how threats to reproducibility, recognized but unaddressed for decades, might finally be brought under control. via
'Rein in the four horsemen of irreproducibility' by from via
Great Nature piece by why/how most scientific research past 60 yrs has been so unscientific. But I disagree with her optomism re future. Big data sets introduce inherent analytic uncertainty. When tortured they'll confess to almost anything
Rein in the four horsemen of irreproducibility
Really nice article from , and an ideal intro to some of the problems of reproducibility in science: "Rein in the four horsemen of irreproducibility".
Rein in the four horsemen of irreproducibility
rein in the four horsemen of irreproducibility by professor
Rein in the four horsemen of irreproducibility - Nature via thanks
Avoiding the “reproducibility apocalypse” - outlines the issues clearly & concisely #reproducibility #researchintegrity #publicationethics
'Rein in the four horsemen of irreproducibility' by from
“Rein in the four horsemen of irreproducibility” by ⁦
Rein in the four horsemen of irreproducibility
Rein in the four horsemen of irreproducibility
Rein in the four horsemen of irreproducibility, from
Rein in the four horsemen of irreproducibility
Rein in the four horsemen of irreproducibility via
Rein in the four horsemen of irreproducibility | Nature