More self-reflection in research can lead to better science A package of papers looking at the social and behavioural sciences shows the value of researchers collaborating to further the cause of reproducible, replicable and robust findings.

A package of papers looking at the social and behavioural sciences shows the value of researchers collaborating to further the cause of reproducible, replicable and robust findings

go.nature.com/4cjtN1X

Investigating the analytical robustness of the social and behavioural sciences - Nature When 100 social and behavioural science claims were examined, 34% of reanalyses closely matched the original results, with 74% reaching the same conclusion, revealing limited robustness of single...

Replication, robustness reproducibility of results across the behavioural + social sciences
www.nature.com/articles/s41...
www.nature.com/articles/s41...
www.nature.com/articles/s41...
www.nature.com/articles/s41...
@balazsaczel.bsky.social @szaszibarnabas.bsky.social @briannosek.bsky.social

📄Published Today in Nature:

500 researchers reproduced 100 studies across the social & behavioral sciences to assess their analytical robustness (led by @balazsaczel.bsky.social & @szaszibarnabas.bsky.social).

Article: www.nature.com/articles/s41...

Preprint: osf.io/preprints/me...

TLDR: 1/11

Really excited to see this published! I contributed a small part by doing one of the "robustness replications" as my econ friends call it. Stellar coordination effort by @balazsaczel.bsky.social, @szaszibarnabas.bsky.social et al.!

Simulation-based validation of Bayes factor computation We propose and evaluate two methods that validate the computation of Bayes factors: one based on an improved variant of simulation-based calibration checking (SBC) and one based on calibration metrics...

Preprint alert: Simulation-based validation of Bayes Factor computation with @paulbuerkner.com and S. Stroppel. We bring lessons learned in SBC to validation of BFs. arxiv.org/abs/2508.11814 The idea is simple: simulate data from the models, fit and see if the inferences are calibrated. 1/10

Our lab has the capacity to test ~500 uni students each semester
If you’re a researcher in cognitive psychology or metascience and need data collection support, we’d love to collaborate. We can help collect high-quality data from a large student sample.
Get in touch to discuss potential projects!

If it's an online test then whatever you have. What do you need?

You can DM or email me

Our lab has the capacity to test ~500 uni students each semester
If you’re a researcher in cognitive psychology or metascience and need data collection support, we’d love to collaborate. We can help collect high-quality data from a large student sample.
Get in touch to discuss potential projects!

www.reuters.com/legal/govern...

See you in Melbourne in Nov 2027, SIPSers and AIMOSers! #metascience

Strategy 4. Integrate analytic quality and expertise. As many-analyst approaches mature, there may be good reasons to introduce more nuance to the wisdom of the crowds principle. Should every analyst’s contribution be weighted equally? How can we account for pre-existing differences between analysts —such as varying levels of expertise in the research topic, methodology and statistics, or differences in the time available to dedicate to the analysis— in a way that supports the overall quality of the project? Rather than solely using these factors as potential exclusion criteria (see e.g., Aczel et al., 2021; Gould et al., 2025), they could also serve as grounds for giving greater weight to submissions from highly rated teams. One could make a strong case for taking factors such as subject-matter and methodological expertise into account when weighing the contributions of individual analysts (see e.g., Aczel et al., 2021). This can be done either when calculating an overall meta-analytic effect size across teams (Bartoš, Hoogeveen, et al., 2025) or by incorporating the weights into a meta-regression analysis

Many analysts are not always equal analysts

"Should every analyst's contribution be weighted equally?"

From "Ten Strategies To Improve Many-Analysts Studies" by Alexandra Sarafoglou, @balazsaczel.bsky.social et al.

Preprint: osf.io/preprints/ps...

#MetaSci

Published today: One of the biggest #science #communication studies to date. We asked 71,922 people in 68 countries how they #engage with information about #science and combined the data with several country-level factors: journals.sagepub.com/doi/10.1177/... #OpenAccess

Core principles of responsible generative AI usage in research - AI and Ethics In a rapidly evolving Generative Artificial Intelligence (GenAI) landscape, researchers, policymakers, and publishers have to continuously redefine responsible research practices. To ensure guidance o...

📚 Core principles of responsible generative AI usage in research link.springer.com/article/10.1...

Make an effect size prediction!

@jamiecummins.bsky.social and I are replicating Balcetis & Dunning's (2010) "chocolate is more desirable than poop" (Cohen's d = 4.52)

Let us known in the replies what effect size you think we'll find. Details of the study in the thread below.

Flyer for the 2025 Visual SciComm Seminar speaker Mike Morrison. A Water Whys logo is at the top left with a question mark embedded in an upside down water droplet. At the center is Mike's picture. The background in the lower half is reminiscent of waves. Friday, October 17, Mike will be the second speaker in the Water Whys Visual SciComm Seminar Series, talking about "Design for Scientists: Help people pay attention to research and love it!" Design and psychology principals are at the core of making visual communication (slides, posters, research reports — anything) engaging. In his presentation, designer and psychologist, Mike Morrison will explore this through games and silly graphics. You’ll see data from eye tracking and other studies beautifully illustrating how the power of good design can make science move faster and be more pleasant to learn (even for scientists). And you’ll get a glimpse into the future of scientific publishing. In the end, you’ll learn immediately-applicable tips that will help you design things that impact more people, and get your science message out further. This FREE virtual talk is from 2-3pm ET / 11am-noon Arizona Time and is open to everyone. Register to get the Zoom link.

@mikemorrison.bsky.social will be sharing how #scientists and #SciComm folks can combine design and psychology principles to create engaging visual science communication tools.

Join this Friday, Oct 17 @ 2pm ET for this FREE webinar.

Sign up for Zoom link. waterwhys.org/seminar/fall...

Multiverse simulation to explore the impact of analytical choices on type I and type II errors in a reaction time study - Behavior Research Methods Researcher degrees of freedom in data analysis present significant challenges in social sciences, where different analytical decisions can lead to varying conclusions. In this work, we propose an exam...

I am happy to announce the publication of our new work on the impact of arbitrary analytical choices on type I and type II error rates. We simulated reaction time data in a conflict task and analyzed the notable CSE effect in a multiverse manner. Worrying results:
link.springer.com/article/10.3...

Transparent peer review to be extended to all of Nature’s research papers From today, all new submissions to Nature that are published will be accompanied by referees’ reports and author responses — to illuminate the process of producing rigorous science.

www.nature.com/articles/d41...

Is it OK for AI to write science papers? Nature survey shows researchers are split Poll of 5,000 researchers finds contrasting views on when it’s acceptable to involve AI and what needs to be disclosed.

www.nature.com/articles/d41...

Led by @ejwagenmakers.bsky.social

[Editorial: Introducing the Journal of Robustness Reports]

We are live!

Introducing the "Journal of Robustness Reports" – a Diamond Open-Access journal dedicated to publishing short reanalyses of empirical findings.

Check out our website and blog post about the journal:
🌐 scipost.org/JRobustRep
📄 www.bayesianspectacles.org/introducing-...

The Journal of Robustness Reports is here. Diamond open access. A model for #openscience and #metascience. @balazsaczel.bsky.social scipost.org/JRobustRep.0... 🧪

Love the idea scipost.org/JRobustRep

"We hope that the Journal of Robustness Reports will help make reanalyses of published findings the norm across the empirical sciences." #diamondOA

scipost.org/JRobustRep/a...

@fbartos.bsky.social
@balazsaczel.bsky.social
@ejwagenmakers.bsky.social

A photo of the textbook 'Discovering statsitics using JASP' on a desk./ The cover has the name in large letters and a drawn image of a futuristic female statuistics professor wearing blue clothes with a circuit-board style print on.

Author copy arrived yesterday. It really was a joy working with @ejwagenmakers.bsky.social and Johnny (is he on here?) on this @jaspstats.bsky.social version of the book.

Publishers trial paying peer reviewers — what did they find? Two journals embarked on efforts to compensate reviewers, with different results.

Two journals tried paying peer reviewers and found increased acceptance, earlier reports, and no change in quality.

WHAT A SURPRISE!
www.nature.com/articles/d41...
@balazsaczel.bsky.social was cited from Eötvös L. Uni. Hungary (where I also teach)

Could libraries band together to ensure open access for all? Through the ‘subscribe to open’ model, libraries’ annual subscriptions ensure that paywalled journals become freely accessible, benefiting researchers and the public alike.

www.nature.com/articles/d41...

🚀Very excited to finally see our paper on nonsignificance misinterpretations published! 📈
Together, @smurphee.bsky.social, Aurelio Fernández, Linda Reimann and I investigated the prevalence of "p > .05 = absence of an effect" interpretations. (1/4)
doi.org/10.1098/rsos...

Don't forget to register to #SIPS2025 !
Early-bird deadline is March 15.

www.improvingpsych.org/SIPS2025/

PSA Around the World online conference with focus on Central n Eastern Europe is in November (6, 14, 22). The abstracts are due March 31st. Open to anyone around the world and on any #philsci topic, but especially on the history and the present of the field in the region. Thanks to the organisers!

We outline how such a computational psychology of science can contribute to #metascience.

Our findings suggest that collective efforts, tailored to group characteristics, may foster behavior change interventions in science, @balazsaczel.bsky.social

👉 doi.org/10.23668/psy...

🧵4/4