A Web crawler is a programs that systematically browses the Web. A Web crawler be called as bots, spider, robot and indexer. Before implementation you need to know type of Crawler.
