A tool to determine if the robots.txt would prevent a given user agent from making a request to a given URI.
James Edward Gray II, Jeremy Friesen, Edgar Gonzalez
July 9, 2010 8:34pm
無