RE: https://fediscience.org/@snakemake/116295568336688286
This is a big step forward: The SLURM plugin for Snakemake now supports so-called job arrays. These are cluster jobs, with ~ equal resource requirements in terms of memory and compute resources.
The change in itself was big: The purpose […]
Did you know? We started to gather cluster profiles for Snakemake users in a little repo: https://github.com/snakemake/snakemake-cluster-profiles
They should serve as a template for others.
#HPC #Snakemake #ReproducibleComputing #SnakemakeHackathon2026
🐍 For all ye #snakemake users out there who want consistent, opinionated formatting of their workflows: we have just released v1.0.0 of snakefmt 🥳
The major update is that it now sorts rule directives (e.g., input, output, resources, params, shell etc.).
See github.com/snakemake/sn... for more
Another terminal based tool to interactively monitor all running #Snakemake workflows is #snkmt: github.com/cademirch/sn....
The modern #Snakemake architecture makes this fairly easy! I just wrote a wasm executor plugin and used the API to build the dag and run the workflow in pyodide. Massive credit to @johanneskoester.bsky.social and the core team for making Snakemake to interoperable.
I learn, that I am the first to write a reporter plugin, which is part of the #Snakemake organization.
That will change. @fbartusch is working on an #ROCrate plugin. Yours truly is working on a #nanopub plugin. Both will ease publishing workflow analysis metadata and making our computing a bit […]
Which new features did I like the most?
Well, there are so many, it merits a preprint for which @egonw already lend tremendous support.
Anyway, here are my favourites:
- #Snakemake tracks all #metadata during the workflow execution. This caused many(!) file access requrests. Now, a SQLite-DB […]
As for the little executor plugin for the #SLURM batch system (for which I promised a release supporting array job support) ... Well, only a little bug fix release could be accomplished: github.com/snakemake/snakemake-exec...
Unfortunately, I wanted to […]
RE: https://fediscience.org/@snakemake/116222696140712833
What a week at the #SnakemakeHackathon2026 !
What a wonderful week with wonderful people!
We were pretty productive and this #Snakemake release is just a peak of it. The list of features, bug fixes, performance improvement and […]
This cannot be:
I am trying to compile a few stats for the #Snakemake executor plugin for #SLURM on #HPC systems. Preparing for a lighting talk at the #SnakemakeHackathon2026
PyPi: 20,000 downloads last month
BioConda: > 60,000 total (aggregated over all versions)
Impressive as it might be […]
An example (dummy) knowledge graph describing a) a Snakemake workflow b) an example data set and c) an example analysis report for this workflow
What do you see here? This is an example knowledge graph describing a #Snakemake analysis workflow. You see the workflow description, a linked data set and a linked report.
All work done to boost #HPC user support for those conducting their workflows on HPC […]
[Original post on fediscience.org]
The #Snakemake plugin for #SLURM on #HPC clusters will support JobArrays, soon:
1057691_1 2dcf44cc-+ rule_map_reads_wild+ 32 COMPLETED 0:0
1057691_2 2dcf44cc-+ 32 RUNNING 0:0
1057691_3 2dcf44cc-+ 32 RUNNING 0:0
1057691_4 2dcf44cc-+ 32 RUNNING 0:0
1057691_5 2dcf44cc-+ 32 RUNNING 0:0
1057691_6 […]
To finish with #haplomics a #snakemake pipeline to run haplotype analysis at any locus , also available on #github
www.medrxiv.org/content/10.1...
On a similar note: there is another (draft) PR. The #SLURM executor plugin for #Snakemake is capable of respecting partition definitions since v. 2.
I had the notion, that this is rather difficult to set this up manually and wrote a little command line helper. It queries the SLURM config and […]
I want to reach out: I have this pending release for the SLURM executor (github.com/snakemake/snakemake-exec... ). It implements better error feedback (in case of hardware failures and otherwise). It would need some thorough checking, and I cannot provoke too many […]
My next obsession: workflow managers 🎈Yesterday night I binged videos from #riffomonas about #reproducible research. He used #snakemake , it looks nice! I also looked up #nextflow It looks fierce! So which one to learn?
Any advice? I'll use it for #bioinformatics #pipeline s; I mainly write in #R
Well timed question. Indeed a student assistant will start on the preparation for being able to have #rust modules in the #Snakemake codebase via pyo3. We will start with the output file index. There might be the first #rust rewrite work already during hackathon in case somebody is interested.
The new #Snakemake releases in the last weeks (9.14.*-9.16) have massively improved the way Snakemake handles source files from github (wrappers, modules, scripts, envs), avoiding the github API entirely and caching (bare) git repos instead. This increases performance and avoid hitting API limits.
Using #easyvvuq to determine the failure probability of a model of a hanging wire and running both models using #snakemake to ensure #reproducibility .
I welcome all critiques! (Hence only a TR 😅)
Sadly it took the STFC library >mo to wrap this as a formal TR and put it online properly! 🙄
MeTAline: enabling reproducible and scalable metagenomic analyses. #MetagenomicAnalysis #ShotgunMetagenomics #Snakemake #Containers #HPC #Bioinformatics #Genomics #NARgenomicsAndBioinformatics 🧪🧬 🖥️
academic.oup.com/nargab/artic...
#Snakemake 9.14 is released. It integrates with yet another scripting language ( #hy, adding to the already existing #python, #r, #rust, #bash, and #julia support). Further it offers a massively improved local storage footprint. Many thanks to our awesome community! snakemake.github.io
This took a while. After the new version of the Snakemake paper (a rolling paper on F1000) came out, the DOI now is "working" 🥳 :
https://doi.org/10.12688/f1000research.29032.3
From my point of view, it particularly describes the working with various #HPC batch systems. And: Development did […]
A worker Snake, symbolizing the Snakemake workflow system, carrying a box labelled "SLURM plugin". Grim Face. Dark cloud overhead, labelled "policy". Subtitle: Reproducible computing on HPC clusters against the odds - the Snakemake SLURM plugin
Just submitted a talk for FOSDEM (been invited). They asked to attach an icon-image for the talk. So I drew one. The compute racks are difficult to identify as such, but this is as far as my aquarelle skills go.
#Snakemake #ReproducibleComputing #HPC
Just getting back from two day teaching #Snakemake programming on #HPC clusters. And I have several observations:
- the education level everywhere else seems way better, than on my home cluster. It is still a good idea, to assume the worst.
- it is never a good idea to accept […]
Finally finished improving the #snakemake workflow for a refutation paper.
Now what's the "law" governing how much time is spent on a false claim vs. how much on its verifiable refutation?
I have no experience with submitting papers disproving others so this is gonna be … interesting.
The #Snakemake wrapper repository and #Snakedeploy have received a major update: each wrapper now reports the latest version with changes, and #Snakedeploy can automatically upgrade all wrappers in your workflow to those versions. snakemake-wrappers.readthedocs.io
#Snakemake has a novel feature for generalizing rules and modules: pathvars! snakemake.readthedocs.io/en/stable/sn...
Pathvars are particularly useful to make modules parameterizable and more generic. Further, they reduce boilerplate code commonly used for defining output folders.
This spring, we had a wonderful time at the CERN shaping the future of the Snakemake Workflow Management System during the Snakemake Hackathon. Next spring we will meet in Munich!
If you want to take part in the Snakemake development, you can still register here […]
Want to help shaping the future of #Snakemake? Then participate in the Snakemake hackathon 2026 in Munich! There's still room as the maximum number of participants is not yet reached. You can register here: indico.cern.ch/e/snakemake-...
#sciworkflows #reproducibility
> Your abstract 'Live Demo: From Snakemake Run to Scholarly Record - Automatic Nanopublications & Workflow Run RO‑Crate Generation' has been successfully submitted. It is registered with the number #150. You will be notified by email with the submission details.
I have quite some work to do […]
