RE: https://fediscience.org/@snakemake/116295568336688286
This is a big step forward: The SLURM plugin for Snakemake now supports so-called job arrays. These are cluster jobs, with ~ equal resource requirements in terms of memory and compute resources.
The change in itself was big: The purpose […]
Did you know? We started to gather cluster profiles for Snakemake users in a little repo: https://github.com/snakemake/snakemake-cluster-profiles
They should serve as a template for others.
#HPC #Snakemake #ReproducibleComputing #SnakemakeHackathon2026
RE: https://fediscience.org/@snakemake/116222696140712833
What a week at the #SnakemakeHackathon2026 !
What a wonderful week with wonderful people!
We were pretty productive and this #Snakemake release is just a peak of it. The list of features, bug fixes, performance improvement and […]
The #SnakemakeHackathon2026 just started! Lots of interesting topics. If you are curious: #SnakemakHackathon or #SnakemakeHackathon2026 are your hashtags to follow. 😉
#reproducibleComputing #OpenScience #Datananalysis
An example (dummy) knowledge graph describing a) a Snakemake workflow b) an example data set and c) an example analysis report for this workflow
What do you see here? This is an example knowledge graph describing a #Snakemake analysis workflow. You see the workflow description, a linked data set and a linked report.
All work done to boost #HPC user support for those conducting their workflows on HPC […]
[Original post on fediscience.org]
I want to reach out: I have this pending release for the SLURM executor (github.com/snakemake/snakemake-exec... ). It implements better error feedback (in case of hardware failures and otherwise). It would need some thorough checking, and I cannot provoke too many […]
This took a while. After the new version of the Snakemake paper (a rolling paper on F1000) came out, the DOI now is "working" 🥳 :
https://doi.org/10.12688/f1000research.29032.3
From my point of view, it particularly describes the working with various #HPC batch systems. And: Development did […]
A worker Snake, symbolizing the Snakemake workflow system, carrying a box labelled "SLURM plugin". Grim Face. Dark cloud overhead, labelled "policy". Subtitle: Reproducible computing on HPC clusters against the odds - the Snakemake SLURM plugin
Just submitted a talk for FOSDEM (been invited). They asked to attach an icon-image for the talk. So I drew one. The compute racks are difficult to identify as such, but this is as far as my aquarelle skills go.
#Snakemake #ReproducibleComputing #HPC
This spring, we had a wonderful time at the CERN shaping the future of the Snakemake Workflow Management System during the Snakemake Hackathon. Next spring we will meet in Munich!
If you want to take part in the Snakemake development, you can still register here […]
Where will I be in early March 2026?
In Stuttgart! At the deRSE conference. I intend to submit a couple of work items dealing with my favourite workflow management system. And the call for contributions is open: https://mastodon.social/@de_rse/115270954346336457
To give you an idea of what I […]
Remember that I have been posting about the #SnakemakeHackathon2025 ?
I never really finished that series. But now, we have two late contributions by Ward Deboutte and @johanneskoester . One describing the polishing of the multiple extension handling of #Snakemake for named inputs ( […]
Hi data analyists from Mainz,
We present a survey that explores how researchers approach computational reproducibility, especially when working with HPC and cloud infrastructure. The goal is to better understand current practices, challenges, and needs around reproducible research workflows […]
Dear HPC users,
A while ago, I posted about a survey that explores how researchers approach computational reproducibility, especially when working with HPC and cloud infrastructure. The goal is to better understand current practices, challenges, and needs around reproducible research workflows […]
First day at the #NESColLab comes to a conclusion (work-wise at least). I have a half-working solution .... and will keep the suspense until a first release is ready. 😉
#hackathon #ReproducibleComputing #fairdatamanagement
@brembs we are conducting a study in the HPC world about reproducibility: https://fediscience.org/@rupdecat/114942626807831429
Are people concerned? If they want to do research with reproducibility in mind, what is keeping them from conducting an at least good-enough approach? (We all know that […]
Interested in HPC compliant data analysis workflows?
I am offering an NHR (German association for HPC resources) course for building and using Snakemake workflows on HPC clusters in Mainz, Germany! Two days: 9. & 10. December 2025 on-site.
To find out more and perhaps enrol, visit the course […]
My first request to the #boost mailing list in years. I hate it, when libraries "suddenly" become deprecated and at the same time stop working altogether.
All I wanted was first to have a working example for a student to work on. 🙄
#ReproducibleComputing #Cpp
The #isc25 is over and I half-recovered from the weekend, too. Time to continue my thread summing up the #SnakemakeHackathon2025 !
To me, an important contribution was from Michael Jahn from the Charpentier Lab: A complete re-design of the workflow catalogue. Have a look […]
@dryak returning from the #isc25 I will continue this thread with something applicable everywhere, not just on #HPC clusters:
Workflow runs can crash. There are a number of possible reasons. Snakemake offers a `--rerun-incomple` flag (or short `--ri`) which lets a user resume a workflow.
This […]
Today tooting from the #ISC25 - the International Supercomputing Conference. What better opportunity to brag about something I've done to facilitate using GPUs with Snakemake?
Here is my contribution, simpler job configuration for GPU jobs:
https://doi.org/10.5281/zenodo.15551797
Not alone […]
This morning, I am travelling to the #isc25 and hit a minor bug on #researchequals. Hence, no updates in the collection.
But still a few to describe without adding the latest contributions:
For instance, this one by Filipe G. Vieira: a helper function to extract checksums from files to compare […]
Before I continue uploading - and I do have a couple of more contributions to add to the #ResearchEquals collection - first another contribution by Johanna Elena Schmitz and Jens Zentgraf made at the #SnakemakeHackathon2025
One difficulty when dealing with a different scientific question: Do I […]
Let's take a look at another contribution of Johanna Elena Schmitz and Jens Zentgraf from the #SnakemakeHackathon2025
Snakemake users probably know that
`$ snakemake [args] --report`
will generate a self-contained HTML report. Including all plots and #metadata a researcher's heart longs for […]
One important feature implemented in the #SnakemakeHackathon2025 : Snakemake will calculate file checksums to detect changes. If a file changes, the rule producing it needs to be re-executed when a workflow it re-triggered. But what if a file is too big for reasonable checksum calculation? You […]
One important bug fix during the #SnakemakeHackathon2025 : the config replacement. Now, users can overwrite existing configurations entirely with `--replace-workflow-config`.
Details: https://zenodo.org/records/15479268
More at https://www.researchequals.com/collections/hm1w-cg/
#Snakemake […]
Did you know? During the #SnakemakeHackathon2025 we had a staggering 194 work items!
It took a while, but now we are gathering contribution reports and present them online as a ResearchEquals (fediscience.org/@ResearchEquals@akademie... collection […]
Have been on a little holiday and meanwhile my pull request for the HPC Certification Forum (#HPCCF - we just _need_ a hashtag 😉 ) on workflow management systems has been approved! Thanks to @jkunkel1 !
To me, it is important that the #HPC community realizes the value of automation in data […]
The #SnakemakeHackathon2025 just induced another collaboration for me!
It is always good to work along with like-minded people, interested in #ReproducibleComputing, #HPC and #Bioinformatics
#OpenScience
Today is the day of closed pull request for #Snakemake. The #SnakemakeHackathon2025 participants worked at full speed!
We decided to write a white-paper summarizing our achievements rather than posting individual things. Suffice to say, that also the documentation made a great leap towards […]
Majority of all participants to the hackathon gathered for a photo at the CERN.
#SnakemakeHackathon2025 ! We started!
At the CERN for better #ReproducibleComputing and #ReproducibleResearch .
