What is Crawler Based Search Engine

Crawlers use spider, crawler, robot and bot programs to crawl and index new content in the search engine's database. It has to follow all these rules for converting any website into search result Crawling, Indexing, Calculating Relevancy and Retrieving Result. Google, Bing, Yahoo are the prime examples of crawler search engines.