- All Implemented Interfaces:
- Configurable, FetchSchedule
public class DefaultFetchSchedule
- extends AbstractFetchSchedule
This class implements the default re-fetch schedule. That is, no matter
if the page was changed or not, the
unchanged, and the updated page fetchTime will always be set to
fetchTime + fetchInterval * 1000.
- Andrzej Bialecki
fetchTime on a
successfully fetched page.
|Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
public CrawlDatum setFetchSchedule(Text url,
- Description copied from class:
- Sets the
fetchTime on a
successfully fetched page. NOTE: this implementation resets the
retry counter - extending classes should call super.setFetchSchedule() to
preserve this behavior.
- Specified by:
setFetchSchedule in interface
setFetchSchedule in class
url - url of the page
datum - page description to be adjusted. NOTE: this instance, passed by reference,
may be modified inside the method.
prevFetchTime - previous value of fetch time, or 0 if not available.
prevModifiedTime - previous value of modifiedTime, or 0 if not available.
fetchTime - the latest time, when the page was recently re-fetched. Most FetchSchedule
implementations should update the value in @see CrawlDatum to something greater than this value.
modifiedTime - last time the content was modified. This information comes from
the protocol implementations, or is set to < 0 if not available. Most FetchSchedule
implementations should update the value in @see CrawlDatum to this value.
state - if
FetchSchedule.STATUS_MODIFIED, then the content is considered to be "changed" before the
FetchSchedule.STATUS_NOTMODIFIED then the content is known to be unchanged.
This information may be obtained by comparing page signatures before and after fetching. If this
is set to
FetchSchedule.STATUS_UNKNOWN, then it is unknown whether the page was changed; implementations
are free to follow a sensible default behavior.
- adjusted page information, including all original information. NOTE: this may
be a different instance than @see CrawlDatum, but implementations should make sure that
it contains at least all information from @see CrawlDatum}.
Copyright © 2012 The Apache Software Foundation