- cross-posted to:
- programmerhumor@lemmy.ml
- cross-posted to:
- programmerhumor@lemmy.ml
You must log in or register to comment.
I just recently seen a python scraper in my server logs earlier today. Strangest thing to see.
As long as the scrapers follows robots.txt
It’s equivalent to “the code.”
It really should be “parlay.txt”.
I feel like there should be a third box with Wall Street raider types, for scrapers that use Selenium browser automation.
I don’t think it’s entirely unblockable - adsense seems to know to only serve unmonetized PSA ads - but I think it’s very difficult to discriminate between “this is a real browser controlled by an end user” and “this is a real browser being controlled by automated test software”.
Fourth panel as well, with those bots collecting data for AI training that don’t respect your robots.txt, change user agents and overload your servers
War boys from Fury Road?
beautiful soup
