S3Mirror is an open source application for transferring large genomic datasets b/t S3 buckets efficiently, reliably, & with full observability.
Built on DBOS Transact, it was created to meet the needs of a pharmaceutical organization handling clinical trial data.
www.biorxiv.org/content/10.1...
I'm trying to get into the habit of producing more video, so on the spur of the moment I made a 7 minute video showing how I used Claude Code to put together a simple GitHub Actions workflow www.youtube.com/watch?v=VC6d...
I think "context engineering" is going to stick - unlike "prompt engineering" it has an inferred definition that's much closer to the intended meaning, which is to carefully and skillfully construct the right context to get great results from LLMs simonwillison.net/2025/Jun/27/...
My notes on Gemini CLI, including poking around in their system prompt which I've extracted into a more readable rendered Gist simonwillison.net/2025/Jun/25/...
As I'm evaluating XML output I keep running into invalid XML slop. So now I'm parsing AI XML slop with an AI generated XML slop parser. This library is 100% Claude generated. github.com/mitsuhiko/sl...
Ditto is probably the most mature sync engine out there used in airlines and other critical environments.
Loved this conversation with Adam who’s the founder of Ditto and previously built Realm.
Lots of great stories and insights in this episode!
It's live now! docs.dbos.dev/llms.txt
Here's video, slides and a detailed annotated transcript from my talk at this week's AI Engineer World's Fair conference in San Francisco - "The last year six months in LLMs, illustrated by pelicans on bicycles" simonwillison.net/2025/Jun/6/s...
This looks great!
Try using models to critique each other, for example, develop your prototype in Claude 4.0 Opus and then use Gemini 2.5 Pro to provide a critique of its work. We’ve had good results using models to critique each other in this way.
I think DBOS needs LLM friendly docs, see eg.: llms.motion.dev
Build collaborative applications with CRDTs straight outta Postgres using the new Yjs Electric provider. 🧵
www.npmjs.com/package/@ele...
We absolutely need a local first podcast episode about this!
Introducing LiveStore (beta)
LiveStore is a next-generation state management framework based on reactive SQLite and built-in sync engine.
livestore.dev
After 4 years of work, I'm incredibly excited to introduce LiveStore, the next-gen data layer I'm building for Overtone.
It's based on reactive SQLite and has a built-in sync engine. Give it a try - would love your feedback!
Shopify Polaris is now built on web components. My team and I have been working on this for a while and it's been a lot of fun to "embrace the platform"!
shopify.dev/beta/next-ge...
Lots of things learned, lots of fun edge cases discovered (hello moving web components to iFrames in Firefox)
Hey look! I've been working on a thing!
It's a statically typed, JIT compiled, hot-reloadable, embedded scripting language.
blog.nlnetlabs.nl/introducing-...
With workflows stored in Postgres, you can:
- Query failed billing workflows
- Identify affected customers
- Fork workflows to create new ones starting from the failed step
Learn more 👇🏽
www.dbos.dev/blog/why-wor...
Need a weekend challenge?
Igor Konnov sets out to specify the Two-Phase Commit Protocol — using nothing less than the Lean Theorem Prover
Modeling distributed protocols has never looked so formal.
protocols-made-fun.com/lean/2025/04...
Announcing Datasette for Newsrooms - a hosted version of @datasette.io specifically targeted at newsrooms and data journalists
Think of it as a library for your data - load in CSVs and JSON, extract data with LLMs, collaborate on analysis with the rest of your team
simonwillison.net/2025/Apr/24/...
New blog post on the mental model I've used when working through complex or confusing papers on transactional systems.
transactional.blog/b...
@taran.space and more friendly folks ARK Builders wrote an article about me, comparing me with alternatives like libp2p, SyncThing and Nebula:
ark-builders.medium.com/the-deceptiv...
Thank you for the kind words :)
Graft is a really interesting new open source replication system designed for read-write multi-leader replication, with a SQLite extension as the first demonstration of what it can do simonwillison.net/2025/Apr/8/s...
Screenshot of the paper https://hyperloop-rails.github.io/220-HowNotStructure.pdf
When your application runs into performance issues, don't be so quick to blame the database — most of the time, the real culprit is your app is using it wrong, especially when ORMs are involved.
🔗 hyperloop-rails.github.io/220-HowNotSt...
Planet Postgres: notso.boringsql.com/posts/know-t... Fantastic article all about timestamps, intervals, and other time-related gotchas. #postgres #planetpg #postgresql (tl;dr always use timestamptz, never timestamp)
A few days ago I gave a keynote at the PaPoC workshop on Byzantine Eventual Consistency and Local-first Access Control. It wasn't recorded, but slides are here
speakerdeck.com/ept/byzantin...
This might be the most incredible software library landing page I’ve ever seen. Totally blown away.
A graphic with a rendering of a custom single board computer with the form factor of a Raspberry Pi, but with different connectors. The text says "Coming soon to a GitHub repo near you"
Ever wanted to build a custom embedded device but felt stuck with off-the-shelf boards?
Meet Trellis! 🌱 A Nerves-first, open-source hardware platform that gives you a solid starting point for your custom hardware projects
I'm beyond excited to speak about it at @elixirconf.bsky.social EU in May!
