robotstxt 0.5.4

Robotstxt Parser allows you to the check the accessibility of URLs and get other data. Full support for the robots.txt RFC, wildcards and Sitemap: rules.

Gemfile:
=

install:
=

Versions:

0.5.4 February 13, 2010 (7.5 KB)
0.5.3 February 13, 2010 (7.5 KB)
0.5.2 December 19, 2009 (7.5 KB)
0.5.1 December 06, 2009 (5 KB)
0.5.0 December 06, 2009 (5 KB)

Development Dependencies (2):

echoe ~> 3.1

rake ~> 0.8

Show all transitive dependencies

Owners:

Authors:

Simone Rinzivillo

SHA 256 checksum:

=

Total downloads 30,564

For this version 16,762

Version Released:
February 13, 2010 8:43pm

Licenses:
N/A

Required Ruby Version: None

Required Rubygems Version: >= 1.2

Links:

Homepage Source Code Download Review changes Badge RSS Report abuse Reverse dependencies