Friends, I have written you a book on forensic metascience.
It is free. You can have it. Happy St. Valentine's Day.
If you wish to give me a gift back, you can use it to cause trouble - the greatest gift of all.
open.substack.com/pub/jamescla...
Call for Proposals: Metascience 2025
Be part of the biggest Metascience meeting yet! We're inviting proposals for:
✅ Virtual symposia
✅ In-person panels
✅ Talks & posters
⏰ Submit by 7 Feb 2025
🔗 Full details: metascience.info/call-for-pro...
Wouldn't it be nice if u could read a published pharma-sponsored clinical trial about some new drug & be confident that the results are what they appear to be?
@chrisdc77 & I think it's very achievable using the @RegReports peer review model.
Our proposal ⬇️ www.openpharma.blog/blog/transpa...
Happy Friday!
Following on from my guide on preregistration, here is the second exciting instalment:
“Registered Reports: A How-To Guide” 📖🔓
osf.io/preprints/ps...
#OpenScience #OpenResearch #AStudentsGuide #Metaresearch #ResearchTraining #RegisteredReports #Preregistration #AcademicSky (1/6)
A first starting pack for metaresearch and open science now counts 150 accounts. I will star a second for those who are not in (sorry but it is difficult to be exhaustive). Please jus answer 👋 to this post.
go.bsky.app/R8RNJgc
Oops, well in that case thanks :)
Would you mind adding me please? Thanks!
We're excited to announce a major update to the Transparency and Openness Promotion (TOP) Guidelines—TOP 2025! These guidelines streamline research practices and provide greater clarity around openness in research.
The preprint is now available. Your feedback is welcome! www.cos.io/blog/new-pre...
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery Chris Lu1,2,*, Cong Lu3,4,*, Robert Tjarko Lange1,*, Jakob Foerster2,, Jeff Clune3,4,5, and David Ha1, *Equal Contribution, 1Sakana AI, 2FLAIR, University of Oxford, 3University of British Columbia, 4Vector Institute, 5Canada CIFAR AI Chair, Equal Advising One of the grand challenges of artificial general intelligence is developing agents capable of conducting scientific research and discovering new knowledge. While frontier models have already been used as aids to human scientists, e.g. for brainstorming ideas, writing code, or prediction tasks, they still conduct only a small part of the scientific process. This paper presents the first comprehensive framework for fully automatic scientific discovery, enabling frontier large language models (LLMs) to perform research independently and communicate their findings. We introduce The AI Scientist, which generates novel research ideas, writes code....
Taylorism is a management philosophy based on using scientific optimization to maximize labor productivity and economic efficiency.
Here's the result of making the false Taylorist assumption that the output of scientific research is scientific papers—the more, faster, and cheaper, the better.
Exciting news: I will be teaching a 4-day summer school on open science this August!
The course is for ECRs in the social & behavioural sciences who want to get up to speed with open & reproducible practices and/or learn how to handle specific problems.
utrechtsummerschool.nl/courses/data...
The Benefits of Preregistration and Registered Reports: http://osf.io/dqap7/
Very pleased to share our new paper "A registered report survey of open research practices in psychology departments in the UK and Ireland" 🎉
doi.org/10.1111/bjop...
We examined open research practices and self-reported capability, opportunity, and motivation to engage in open research practices.
“While both replication and theory crises have caused concern for scientists and philosophers alike, the reforms that so far have resulted allow for cautious optimism.”
New chapter by Tomos ap Sion & @briandavidearp.bsky.social
www.researchgate.net/profile/Bria...
#MetaSci #PhilSci #Methodology
New blog post! "Preregistration: More Promises than Pitfalls," some thoughts inspired by the recent meeting at the Royal Society. I don't find many pitfalls with preregistration itself, but there are plenty of problems with how we talk about it. getsyeducated.substack.com/p/preregistr...
New Managing Editor position!
COS is conducting a research and development project on a new model of scholarly communication called Lifecycle Journals to evaluate its strengths and weaknesses and suitability for transition into a scalable and sustainable solution.
jobs.lever.co/cos/1db59b80...
Very interesting and encouraging pilot at the University of Sussex — hope this can spread to other departments and universities!
h/t @deevybee.bsky.social
www.ukrn.org/2024/01/30/p...
Screenshot of survey front page, explaining its aims, participant information, ethics, and contact information. Survey aim reads: "This survey aims to look at the extent to which the results of a research study influence publication in the peer-reviewed literature. The survey should take between 5 to 10 minutes to complete, depending on the answers you provide."
Are you a researcher?
Please participate in our global survey (5-10mins) and share it with other researchers in your network - thanks! 🙏
➡ cardiffunipsych.eu.qualtrics.com/jfe/form/SV_...
#AcademicSky #metasci #PhDSky #PsychSkySci #STS 🧪
Advert for a Royal Society Meeting on "The promises and pitfalls of preregistration" (4-5 March, 2024). Organised by Tom Hardwicke, Marcus Munafó, Sophia Crüwell, Dorothy Bishop, and Eric-Jan Wagenmakers
A preliminary agenda is available for our meeting on "The promises and pitfalls of preregistration". We've also extended the submission deadline for lightning talks and posters to 29th January. Registration and submission details on the meeting website: royalsociety.org/science-even...
Our new review article in Current Opinion in Psychology -- Interventions to Counter Misinformation: Lessons from the Global North and Applications to the Global South authors.elsevier.com/a/1iDnO_,5MJ...
Part of special issue on "The Psychology of Misinfo." www.sciencedirect.com/journal/curr...
New preprint: When and How to Deviate from a Preregistration osf.io/preprints/ps... I hope this helps researchers to reflect on the consequence of deviations from preregistrations by evaluating the test's severity and the validity of the inference.
We’ve spent the last decade talking a lot about replications, but to what extent has this talk translated to a tangible change in how often replications are published in psychology journals? osf.io/preprints/ps... /w Katherine Lee, Sarah Schiavone, @mijke.bsky.social @simine.com (1/9)
I’m a fan of trying to quantify how often reviewers & editors check authors’ preregistrations (eg, by @syeducation.bsky.social), but also we should not forget that reviewers fail to check for deviations from a priori plans in 100% of unpreregistered studies. Because they can’t.
Remember best practice in a power analysis is to plan not for the effect you expect, but for the smallest effect size of interest. The type 2 error is a conditional inference: If there is an effect I care about, I would have X probability to detect it. lakens.github.io/statistical_...
Finally! New policies for Psych Science in 2024 (part 1):
Editorial 1:
www.psychologicalscience.org/publications/psychologic...
Editorial 2 w/ @tomhardwicke.bsky.social
:
www.psychologicalscience.org/publications/psychologic...
New preprint! How often do editors/reviewers actually compare preregistrations to the reporting in the article? Almost never. This figure sums up all of the results. Not good.
Interesting stuff! Reminds me of this paper: doi.org/10.1371/jour...
175: Defending against the scientific dark arts
We chat about a recent post from @deevybee.bsky.social, in which she proposes a Master course that will provide training in fraud detection—what should such a course teach & where would these graduates work?
everythinghertz.com/175
Screenshot of main website dashboard, listing journals which offer (or have offered) the Registered Reports manuscript format. Author/reviewer ratings across the speed and quality categories are aggregated, allowing website users to sort by either category.
📢 Have you authored or reviewed a Registered Report?
✏️ Please give feedback about your experience (takes 5-10mins):
registeredreports.cardiff.ac.uk/feedback
📊 So far authors & reviewers have given 345 pieces of feedback across 73 journals - see our dashboards!
#AcademicSky #metasci 🧪
Screenshot of survey front page, explaining its aims, participant information, ethics, and contact information. Survey aim reads: "This survey aims to look at the extent to which the results of a research study influence publication in the peer-reviewed literature. The survey should take between 5 to 10 minutes to complete, depending on the answers you provide."
Calling all researchers - please participate in our survey (5-10mins) and share with your networks - thanks! 🙏
➡ cardiffunipsych.eu.qualtrics.com/jfe/form/SV_...
#AcademicSky #metasci #PhDSky #PsychSkySci #STS 🧪
Registered Reports in Sport, Exercise, and Performance Psychology (Editorial): http://osf.io/rs4t2/