A queue-based fetcher.
Splits FetcherOutput entries into multiple map files.
This class picks items from queues and fetches the pages.
This class described the item to be fetched.
This class handles FetchItems which come from the same host ID (be it a proto/hostname or proto/IP pair).
Convenience class - a collection of queues that keeps track of the total number of items, and provides items eligible for fetching from any queue.
This class feeds the queues with input items, and re-fills them as items are consumed by FetcherThread-s.
Copyright © 2016 The Apache Software Foundation