Following up from @drjbeaudry.bsky.social's wonderful recap email yesterday, here is the group photo from #AIMOS2025. If you couldn't make this this year, we hope to see you next year in beautiful Wellington!
Last week, #AusHSI PhD student @alexdgibson.bsky.social presented at #AIMOS2025, highlighting a new global issue of unreliable data within #clinicalprediction model research and clinical practice, which has potential to influence patient outcomes and evidence-based decisions. @aimosinc.bsky.social
What an amazing conference! Huge thanks to @drjbeaudry.bsky.social, Joanna Diong, and committee for organising. Looking forward to next year! #AIMOS2025
Thanks to everyone who came to Sydney for #AIMOS2025! I can't wait to see you in Wellington, NZ next year!
James Heathers holding a bouquet of flowers.
What do you do when you get flowers that you can’t take home on a plane? You give them to James Heathers (for his Mum). @jamesheathers.bsky.social
#AIMOS2025
That’s a wrap! #AIMOS2025
The speaker with the closing slide showing AIMOS' logo and the dates of the conference
Jen Beaudry in front of a slide with the previous conference chair in front of the previous conference chair. So meta!
Dr. Jen Beaudry @drjbeaudry.bsky.social is closing the #AIMOS2025 conference, pointing out new and existing fellowship and funding opportunities for meta-research projects. "We are having a Meta Moment of $6.5 million dollars!"
We end with a very meta photo of the conference chairs!
Thank you!
NC: Industry-funded papers underestimate the risk.
We scientists need to shape the public narrative around the harms of their products - we should get out of our echo chambers. There is wide partisan support to have chemicals regulated by government. We need to move the needle.
#AIMOS2025
NC: Another systematic review came to very different conclusions, but ... it was funded by the American Chemistry Council.
We need more actual systematic reviews that do not have those conflicts of interest -
LInks:
prheucsf.blog/category/op-...
www.industrydocuments.ucsf.edu/home/
#AIMOS2025
Slide: EPA Relied on flawed systematic review methods and an outdated literature search
NC: EPA released a draft review that looks great! Systematic review! But it is very flawed. They only included papers older<2019, excluded 100s of relevant studies, it was cherry picking data.
www.epa.gov/assessing-an...
We did a systematic review: pubs.acs.org/doi/10.1021/...
#AIMOS2025
NC: TSCA: primary federal law regulation production/use of non-pesticide chemicals. But after Obama admin, industry people were put in power and chemical lobby were fighting regulation.
EPA has failed to rely on science for their evaluation.
en.wikipedia.org/wiki/Toxic_S...
#AIMOS2025
Speaker and slide stating "but the reality is..." with pictograms of all the chemicals we are exposed to
NC: We know this is true, we have preclinical/animal data, but we are told to prove this through RCTs.
US EPA released reports but Australian government still claims PFAS are not proven to be bad.
We are exposed to 1000s of chemicals, yet we still evaluate one chemical at the time.
#AIMOS2025
NC: We are more and more exposed to plastic - oil companies love to produce more as we move away from fossil fuels.
We all have exposure signals in our blood. Pollution kills more people than drugs, alcohol, or smoking.
Low-income people more exposed.
#AIMOS2025
The talk starts with a video about PFAS (toxic chemicals) leakage from a chemical plant near Jervis Bay.
More background: www.abc.net.au/news/2023-02...
Leading organizations did not use a lot of systematic reviews about environmental hazards.
www.sciencedirect.com/science/arti...
#AIMOS2025
The speaker at the lectern
We will end the @aimosinc.bsky.social conference with a plenary session by Nicholas Chartres, with "Corporate Vectors of Chronic Disease: Using Internal Industry Documents to Safeguard Environmental Health Research Integrity"
#AIMOS2025
I probably won't have time to post much about this, as we are going to do some R'ing ourselves on our laptops!
#AIMOS2025
Workshop Speaker/Chair = Annie Whamond Topic/Title = Retrieving Scholarly Metadata with R: Introduction to Crossref and OpenAlex APIs Outline = Application Programming Interfaces (APIs) are useful tools that allow users to retrieve large batches of metadata with relative ease. Accessing these programmatically, rather than through a web browser interface, enhances research reproducibility and replicability. This introductory workshop focuses on two major databases for scholarly publishing, Crossref and OpenAlex, that have APIs accessible via R packages without paywall barriers. Participants will be guided through how to: * Use these APIs responsibly * Retrieve and explore batches of article metadata * Understand the different strengths and limitations of both databases * Fetch citing and cited-by data for specific articles of interest * Define additional parameters for answering specific research questions RMarkdown Scripts with instructions and exercises will be provided, so no prior programming or API knowledge is required. However, it is essential that attendees bring a laptop with R and RStudio already installed if they wish to actively participate. We also strongly recommend pre-installing packages "rcrossref" and "openalexR" in case internet issues arise on the day. All workshop resources, including scripts and dummy data, will be made available to all AIMOS attendees via GitHub.
Good morning from Sydney, on day 3 of the @aimosinc.bsky.social conference.
This morning I'll attend a workshop by the amazing @anniewham.bsky.social on: "Retrieving Scholarly Metadata with R: Introduction to Crossref and OpenAlex APIs"
#AIMOS2025
Unfortunately I couldn’t be there in person for my #AIMOS2025 talk, but I’m grateful for the chance to share what we’re building at @sci-integrity.com, especially alongside @aimosinc.bsky.social, one of our wonderful partners!
🌏🤝 #SIA #ResearchIntegrity #OpenScience #ScienceIntegrityAlliance
More people need to know about MetaROR @metaror.bsky.social! If you are doing meta-research, then you get that the for-profit publishing system is a headache – you can help sideline journals by having your preprints handled by the MetaROR team for the peer review process. #AIMOS2025
So much energy and laughter in a Jeopardy-style game about Responsible Research Assessment with DORA. Thanks @ginnybarbour.bsky.social @dorassessment.bsky.social
#AIMOS2025
Q: Why is international collaboration essential for advancing research?
A: Understand global initiatives
No: It promotes shared standards
#AIMOS2025
Q: What training should judging panels receive for assessing awards and grants fairly?
A: Onthe principles responsible research assessment
No: Training to avoid hidden biases and to consider diverse research contributions
#AIMOS2025
Responsible-Research-Assessment-Jeopardy cracks me up. We are losing, but possibly laughing the most. #AIMOS2025
Q: "What is one way to make hiring and promotion decisions fairer?"
A: Give committee members the right tools
No: Use transparent standard criteria for all applications
#AIMOS2025
This is a Jeopardy game where the different tables can earn 'money' by answer open-ended questions
(the questions are hard to understand, some good answers are valued as incorrect, it's about being able to cough up someone's idea of the correct answer, and this makes me very insecure).
#AIMOS2025
Our current systems for research assessment are narrow and flawed - @ginnybarbour.bsky.social
@aimosinc.bsky.social
#AIMOS2025
Hackathon Speaker/Chair = Ginny Barbour Janet Catterall Topic/Title = Hacking the system: improving research assessment by stealth Outline = The aims of this session are to: 1. Ensure participants are familiar with key principles of responsible research assessment 2. Understand the link between responsible research assessment, open science and research quality 3. Take away 2-3 concrete actions that participants can use to ""hack"" research assessment at their institution Many researchers are keen to see improvements in research assessment in order for it to support open and reproducible research. However, it is often the case that researchers don‚Äôt know where to start with change at their institution, and may feel relatively powerless in doing so. Using the new guide from DORA - A Practical Guide to Implementing Responsible Research Assessment at Research Performing Organizations (https://sfdora.org/resource/practical-guide/) we will outline key starting points and tools for effecting change. Through group discussions, we will collaboratively develop a set of simple ways that individuals - especially early and mid-career researchers (who are often poorly represented in policy discussions) - can take back to their institutions, and via their own local networks and peer groups help initiate and catalyse responsible research assessment ‚ especially as it relates to open science and research quality.
Slide about DORA
Now attending the Hackaton "Hacking the system: improving research assessment by stealth"
DORA: Valuing academics by a wider range of scholarly products (wider than number of publications/citations), such as sharing datasets, protocols, software.
sfdora.org
#AIMOS2025
Discussion about next steps for MetaROR @metaror.bsky.social A partnership between @aimosinc.bsky.social and @rorinstitute.bsky.social
Led by @alexh.bsky.social and @aidybarnett.bsky.social
#AIMOS2025
Discussion Speaker/Chair = Vincent Mourik Topic/Title = Mitigating risks for early career researchers that blow the whistle on unreliable research Outline = Unreliable research is often best spotted by those that are closest to it: the people working in the lab, those doing the technical work. Typically, these are non-tenured researchers in the first decade of their career. Speaking up, or having to blow the whistle, is particularly risky in this phase of the career. Retaliation by those implicated can take many forms. Others more distant might fear or misunderstand the person and let this cloud their judgement of the person and/or their work. Topics I would like to have a discussion on are: * best whistleblowing practices‚ * overview of risks and their impact, * how and where to seek support, * implementing mitigation strategies such as how to convincingly embed a whistleblowing event into one‚ * career narrative. I would seek a bullet point style few pager aimed at whistleblowing ECRs as an output, to help them forward early on in the process, and view a round table discussion with a few interested others as a good starting point for that. Disclaimer: I blew the whistle myself on a case of unreliable research in quantum physics/computing involving Delft University of Technology, University of Copenhagen and Microsoft. My proposal is grounded in my own experience. No Zoom link available
After a quick tea break, I am in the parallel discussion session: "Mitigating risks for early career researchers that blow the whistle on unreliable research" with chair Vincent Mourik @vincentmourik.bsky.social
(Per request, I will not be posting this discussion).
#AIMOS2025
Speaker and the title slide
4. Jacques Raubenheimer: You get what you pay for & value what you work (hard) for—Avoiding the pitfalls of bad Google Trends research.
Google Trends: everything is shown scaled to the top (100%), based on a random subset. You need to use the API to get non-scaled data.
#AIMOS2025