What is Slurp?

What is Slurp? Inktomi Web Robot Search Engine Crawls Index
Slurp - Inktomi's Web Robot

     Slurp is Inktomi Corporation's web-indexing robot. It collects documents from the web to build a searchable index for search services using the Inktomi search engine, including Microsoft and HotBot.
     Frequency of accesses: Since Slurp crawls billions of pages from the entire web, it uses a large number of systems for web crawling. So your web server may see contacts from a number of different Inktomi crawler client IP addresses. The different crawler systems are coordinated to limit the activity on any single web server. Slurp sends an average of no more than 60 requests per minute to a single web server, and allows no more than one active connection to a single web server. We determine a single web server by IP address, so if your host is serving multiple IPs it may see higher levels of activity.

More on this subject
Beginner's Help
BUG Club Home

What is Slurp? Inktomi Web Robot Search Engine Crawls Index