Group :: Development/Perl
RPM: perl-WWW-RobotRules
Navigation
Main Changelog Spec Patches Sources Download Gear Bugs and FR Repocop
Current version: 6.02-alt1
Build date: 20 february 2012, 16:30 ( 634.5 weeks ago )
Size: 11.90 Kb
Home page: http://search.cpan.org/dist/WWW-RobotRules/
License: GPL or Artistic
Summary: Database of robots.txt-derived permissions
Description:
List of contributors List of rpms provided by this srpm:
ACL:
Build date: 20 february 2012, 16:30 ( 634.5 weeks ago )
Size: 11.90 Kb
Home page: http://search.cpan.org/dist/WWW-RobotRules/
License: GPL or Artistic
Summary: Database of robots.txt-derived permissions
Description:
This module parses /robots.txt files as specified in "A Standard
for Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html>;
Webmasters can use the /robots.txt file to forbid conforming robots
from accessing parts of their web site.
Current maintainer: Alexey Tourbin for Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html>;
Webmasters can use the /robots.txt file to forbid conforming robots
from accessing parts of their web site.
List of contributors List of rpms provided by this srpm:
- perl-WWW-RobotRules