Oh I didn’t realise writes had to go to the remote first. That’s a shame 😔

Oh that’s a super cool idea! I’m familiar with nats so might add that to my todo list of things to explore.

I did a POC to see if it would work and it does. I made a rough implementation of sending the events via a POST request to a different server which then sends them off to relays.

Thank you for the genius idea. It's helping me on my journey. Look forward to seeing what you eventually come up with!

Journey to a distributed PDS: A New Hope

A write up of my Eureka! moment getting my 2 PDS instances sending events to relays correctly.

Oh and I’ll try and go full CAP for the lulz

I’m mainly doing it because I want to host it on my Pi at home but I’ll be damned if I’m going to suffer because my kids accidentally switched the WiFi off 😂

Also PDS embedded in a phone app sounds cool

Yh my thoughts are on point with both your thoughts. I’m taking things steps at a time but my main goal is to be able to have a PDS that is a main PDS but if that were to be shut down, I can switch automatically to another one without having to do anything.

The journey to a distributed PDS - Wills thoughts

Really cool to see @tangled.org natively embedded in @leaflet.pub. Excited to see more experimentation here.

willdot.leaflet.pub/3miiirzf7jc2j

This overcomes the problem I had where routing a xrpc/com.atproto.sync.subscribeRepos request to the right instance of the PDS was failing.

This is basically a POC that @bad-example.com has spoken about, so shout out to them for the idea!

Screen shot of my test account with various posts testing the multi PDS deployment

I did it. It actually works 🥲

I have 2 Cocoon PDS instances running. Instead of pushing events to the websockets to relays, I have a separate service that they post events too.

That separate service then handles the websockets connections the relays connect to and the push the events received..

Lexicon Service Docs Riffing on some ways to make network services more legible to humans and machines

more pizza for your consideration

The journey to a distributed PDS - Wills thoughts

Thank you!

So far it’s just me posting on Bluesky and this leaflet series. Hoping to get part 2 out this week.

willdot.leaflet.pub/3miiirzf7jc2j

Just went to the second page of Google. It was strange.

I’ve never given a presentation at a conference and never really wanted to due to my hatred and fear of public speaking.

But I just day dreamed giving one about distributed PDS’ and let me tell you this… it was incredible. I’d give a good presentation.

Gotta try and figure out what the hell is written in the event that’s sent now.

To the docs!

This post was not only written to the relays websocket connection but also posted to an http server 🙈

30 or 40 mins into the new Super Mario film, this family comes in muttering “I thought the adverts would still be on, we aren’t that late”.

5 minutes later they get up, and walk out. Wrong screen 😂

On the upstream, under the Choose Pull Strategy, you select compare forks.

I’m attempting to run 2 PDS instances behind a Caddy load balancer. But when the request crawl request goes out, I’m prepending a sub domain so I can route the syncRepo request to the correct PDS instance.

All other requests go to the main domain.

Yh you’re right. I tried it and it kinda worked if I routed the sub domain to the PDS in Caddy but there was some weirdness I need to figure out.

Also checking out debug.hose.cam proved that they were treated as different domains.

TBH still not sure if its “illegal” towards spec to do this

When calling com.atproto.sync.requestCrawl does the hostname sent in the request have to exactly match the PDS or could it be a sub domain?

For example: pds404.uk is the host of a PDS but could I request a crawl for one.pds404.uk and have it work? 🤔

I wonder if somewhere one earth, at the right time, if you’ll be able to see the space craft passing the moon 🤔 it’ll probably be the size of a dust spec though

The journey to a distributed PDS - Wills thoughts

Did a quick write up on the work I've been enjoying to make a PDS distributed. #atproto

An app that changes your password to something random and doesn’t reveal it to you until after a period of time has passed.

The journey to a distributed PDS - Wills thoughts

Did a quick write up on the work I've been enjoying to make a PDS distributed. #atproto

willdot.net/cocoon An atproto PDS written in Go

Yh here we go. There’s a branch called Turso-db.

tangled.org/willdot.net/...

I say another use case but thinking about it, it’s the exact thing I was talking about except it’s not a phone app, I’m just distributing my PDS

Either way, I hit the problem sooner than expected

@bad-example.com I’ve found another use case for your firehose inverter you mentioned about 👀

bsky.app/profile/bad-...

Here’s the problem with having multiple instances of a PDS.

Requesting the crawl from one or both asks the relays to call the subscribe repos endpoint. But when you’re load balancing that, the subscribe might not hit the instance that requested the call.

This is the kind of challenge I live for