Business / Search Engine Optimization (SEO) / Crawler: (robot, spider, crawler) A program which performs a task more or less autonomously. Search engines use bots to find and add web pages to their search indexes. Spammers often use bots to "scrape" content for the purpose of plagiarizing it for exploitation by the Spammer.
Business / Internet Marketing / Robot: A file place on your website directory tree which gives instructions to robots/spiders as to what content to access. MORE
Technology / Home Audio / Spider: The rear suspension element on the cone of a loudspeaker. All the moving parts of the speaker are suspended by the spider in the rear and the Surround in the front. The spider is so named because of i MORE
Business / Internet Marketing / Invisible Web: A term that refers to the vast amount of information on the web that isn't indexed by search engines. Coined in 1994 by Dr. Jill Ellsworth. MORE
Business / Internet Marketing / Yahoo Slurp: Yahoo web crawler. Instructions to Yahoo Slurp can be directed in the robots.txt file. Googlebot statistics can be viewed with web analytics software, on webserver logs, and in Google webmaster tools. MORE