Donncha,

Perhaps in your case that works. I’ve not been hit so terribly by libwww-perl on my site. But being a PHP and Perl programmer myself, I regularly write apps for people who are looking to incorporate blog RSS/XML/ATOM feeds into their headlines or sites. Of course of late I’m doing most of that work in PHP, but I’m still doing some sites that prefer to crawl feeds in perl and database them. For my clients, I always set the agent name and it refers back to their site so a webmaster could easily tell that they had been crawled by that particular site, but my point remains that libwww-perl is frequently used to retrieve XML/RSS/ATOM feeds. If you exclude it across the board, you are stopping out some sites from reading your feed.