I tried GLM and Qwen last week for a day. And some issues it could solve, while some, on surface relatively easy, task it just could not solve after a few tries, that Opus oneshotted this morning with the same prompt. It’s a single example ofcourse, but I really wanted to give it a fair try. All it had to do was create a sortable list in Magento admin. But on the other hand, GLM did oneshot a phpstorm plugin
For me, it is working excellent, I almost never check the spam directory for false positives, and it happens maybe once a month for me to receive a spam message in my inbox. I think it is comparable to Gmail, maybe a bit better.
No, since they're simply too many. For an e-commerce site I work for, we once had an issue where some bad-actor tried to crawl the site to set up scam shops. The list of IPs were way too broad, and the user-agents way too generic or random.
Could you not also use an ASN list like https://github.com/brianhama/bad-asn-list and add blocks of IPs to a blocklist (eg. ipset on Linux)? Most of the scripty traffic comes from VPSs.
Thanks to widespread botnets, most scrapers fall back to using "residential proxies" the moment you block their cloud addresses. Same load, but now you risk accidentally blocking customers coming from similar net blocks.
Blocking ASNs is one step of the fight, but unfortunately it's not the solution.
Hypothetically, as a cyber-criminal, I'd like to thank the blacklist industry for bringing so much money into criminal enterprises by making residential proxies mandatory for all scraping.
I’m afraid I don’t know for sure, I only know that the woff2 file I generated with the CLI worked fine in all the browsers I needed it to. Other posters have said that Google may do some user-agent sniffing or other fingerprinting to maybe serve an even more reliable version, but I can’t comment on that.
reply