Just keep going.

In any case, it's important to remember that the CD was just a theoretical framework; something put together when we knew very little, which was useful for stating problems clearly and thus guiding experiments. If we do find things that "break" it, great!

A diagram showing that 24 annually funded grants becomes only 15 funded grants if 50% of the funds for competitive grants are used for multi-year funding.

I was preparing for a talk I gave last evening to a lay audience and came up with this graphic to explain multi-year funding and why it leads to fewer competitive awards and funding investigators and projects.

6/8

This is truly one of the most remarkable advances in cancer therapy in history: metastatic pancreatic cancer was a short-term death sentence and this new approach put many of them into long term remission. This is why we invest in science, and why RFKs effort to kill this work is misguided 🧪

US lawmakers intensify scrutiny of scientific-publishing practices A congressional hearing covered the rise of paper mills and the costs of open-access publishing — but there was little agreement on what reform would entail.

In a rare show of unity, both R and D US lawmakers agree: the scientific publishing industry needs reform.

Lawmakers are worried about the literature being flooded with 'AI slop' and the high open-access fees that some publishers charge.

Read more @nature.com on this week's House Science hearing

Really enjoyed this fantastic conversation with @jontytownson.bsky.social! Give us a listen!

🧪 #metascience

I wish you luck with your experiment. Hope it works out for you.

I don't see the 2025 numbers. The annual report for 2024 says 713 allegations received, 38 cases carried over from the previous year. Of course they are very limited in what they investigate. If one wanted to broaden what they investigate, one would have to increase staff.

Ah. The funder I am most familiar with in the US does have an ORI. ori.hhs.gov

I'm talking about ORI at the funder. Which exists.

Partly this works at software companies because the definition of "error" is straightforward. Code runs or doesn't. Establishing beyond a reasonable doubt that something in a mouse model of cancer is an "error" is much much harder.

But bigger, because claims aren't limited to fraud and misrepresentation.

So anyone can call up the funder and allege error and get paid. Except the people who adjudicate the claims.

I'm sorry, I just cannot see how this works.

In your model, the funder pays the error reporters, and the adjudicators ... adjudicate.

If you reverse that - funder pays the adjudicators, who are expert professionals in handling claims, you've re-invented the Office of Research Integrity.

I think we just have completely different views about this.

One way I understand what you're proposing is that you have people who report what they think are errors; professional adjudicators who decide if the reports are, indeed, errors; and the funder who pays for it all.

Something like the X02? Which I've been told is not popular because it increases administrative burden (requirement to apply to apply).
grants.nih.gov/funding/acti...

Collaboration over competition. All day, every day.

There’s just so many reforms that *sound* like a good idea (and many are!!) but we have to accept that implementing one thing means we can’t afford another.

If you’re not careful you can 5% and 10% (nickel and dime, if you will!) yourself right out of having any money to actually pay for, you know, new science.

Completely agree. I might suggest another way to fund those people would be through an individual award, like the NIH R50. But!! We have to be thoughtful about any new program. It is so, so, so easy to say o it’s just this one little extra thing to support!

I

Importantly, the reward is for working together and not accusing each other. Solution focused, not problem focused. That is an experiment I think could be worth running, and I give it to you free gratis.

Proposal describes the problem, why it's significant and deserves investment to resolve. Run it through review in the regular way. Minimal increase in administrative burden, money as the incentive to work out the bug, competitive proposal process covers the legal bases.

If you want to dedicate 5%of your budget to improving reliability (why 5? too much, too little? what impact does that have on your award rate? a lot, a little?) here's my suggestion: set up a grant program that requires two PIs who have a disagreement to apply together.

If you want to really make positive change, you have to think about implementation and second order effects. In my opinion, in this proposal, I see a lot of hurdles to the former and a lot of negative knock-ons in the latter.

Let's say we get past all of this. Now. What do you see as the implications, in the real world where papers have more than one funder, when one of those multiple funders says a paper is "bugged"? What should they be, and what can we predict is likely to happen?

You can just ask your existing staff to take this on (very common!) but that's going to slow down everything else you care about. Like issuing notices of award, or chasing down every dataset a PI has published to somehow manually inspect and ensure it's sufficiently transparent?

These same staff will also have to answer questions from PIs, from bug hunters, from the adjudicators. They have to referee and direct traffic quite a lot. This is the point I was making about administrative costs and friction. It's not just 5% (if you can figure out how to legally structure that)

And for a small nonprofit, the legal fees to figure it out could be a disincentive.

Assuming you get past that. Now, how do you, as a funder, administer this program? You need staff to "assign those adjudicators" ahead of time. Where do you get them.

Assuming we can get past that: how would the payments be structured. Not as a grant, nor as a contract, both of which require a proposal and a competition to be awarded. I don't know how you would set this up, legally. That's going to cause a large government funder to be averse.

Do the adjudicators get the bug bounty, or does the bug reporter? And if the former, does that not introduce a conflict. If adjudicators don't get part of the bounty, that's unpaid labor, and as I said, we have a hard enough time right now sourcing review expertise.

Much like paying authors based on where they publish leads them to cut corners to get into a journal that brings a payday. I have other practical concerns. You say "adjudicators will be named ahead .. for their fairness" Who decides what's "fair"? How do they decide?

I appreciate your creativity and motivation to incentivize positive culture change. However, I am skeptical that this specific proposal would in fact lead to positive change. My number one concern is that offering people money to find errors is going to incentivize over-reporting.