Web crawler
Noun
Meaning
Web crawler (plural Web crawlers)
(Internet) Synonym of Web spider
Synonyms
Examples
Architectures High-level architecture of a standard Web crawler A crawler must not only have a good crawling strategy, as noted in the previous sections, but it should also have a highly optimized architecture.
By the time a Web crawler has finished its crawl, many events could have happened, including creations, updates and deletions.