I started selfhosting #git around 10 hours ago. And already #openai (WHO AR NOT OPEN ABOUT ANYTHING) has started to crawl it. Taking up about 150MB of data (all text btw). My #cloudflare now uses #aibotmitigation, and I'll add robots.txt. but it's insane the speed they steal your content at.
0
0
0
0