Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
With the rise of AI, web crawlers are suddenly controversial (theverge.com)
5 points by thunderbong 71 days ago | hide | past | favorite | 3 comments


It was happening significantly before the rise of AI. It's even more now.

I am not sure where exactly we are headed through all this, but I feel like overall having data be a shared commons has been beneficial.


Maybe everyone would start putting their content behind a wall. Lol.


That's what I did. I still can't find an effective way of protecting against genAI scrapers while still keeping my sites available without a login.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: