Uses of Interface

Packages that use RobotRules
org.apache.nutch.protocol.file Protocol plugin which supports retrieving local file resources. 
org.apache.nutch.protocol.ftp Protocol plugin which supports retrieving documents via the ftp protocol. 
org.apache.nutch.protocol.http.api Common API used by HTTP plugins (http, httpclient

Uses of RobotRules in org.apache.nutch.protocol

Classes in org.apache.nutch.protocol that implement RobotRules
 class EmptyRobotRules

Fields in org.apache.nutch.protocol declared as RobotRules
static RobotRules EmptyRobotRules.RULES

Methods in org.apache.nutch.protocol that return RobotRules
 RobotRules Protocol.getRobotRules(Text url, CrawlDatum datum)
          Retrieve robot rules applicable for this url.

Uses of RobotRules in org.apache.nutch.protocol.file

Methods in org.apache.nutch.protocol.file that return RobotRules
 RobotRules File.getRobotRules(Text url, CrawlDatum datum)

Uses of RobotRules in org.apache.nutch.protocol.ftp

Methods in org.apache.nutch.protocol.ftp that return RobotRules
 RobotRules Ftp.getRobotRules(Text url, CrawlDatum datum)

Uses of RobotRules in org.apache.nutch.protocol.http.api

Classes in org.apache.nutch.protocol.http.api that implement RobotRules
static class RobotRulesParser.RobotRuleSet
          This class holds the rules which were parsed from a robots.txt file, and can test paths against those rules.

Methods in org.apache.nutch.protocol.http.api that return RobotRules
 RobotRules HttpBase.getRobotRules(Text url, CrawlDatum datum)

Copyright © 2012 The Apache Software Foundation