Business / Internet Marketing / Crawler: A program used by a search engine to 'crawl' links on the Internet to find and index content. Also called a robot or spider. Can be used to identify and differentiate between types of crawlers indexing your site.


Business / Internet Marketing / Robot: A file place on your website directory tree which gives instructions to robots/spiders as to what content to access. MORE


Business / Finance / Bot: Shorthand for bought. Antithesis of SL, meaning sold. MORE


Technology / Home Audio / Spider: The rear suspension element on the cone of a loudspeaker. All the moving parts of the speaker are suspended by the spider in the rear and the Surround in the front. The spider is so named because of i MORE

Invisible Web

Business / Internet Marketing / Invisible Web: A term that refers to the vast amount of information on the web that isn't indexed by search engines. Coined in 1994 by Dr. Jill Ellsworth. MORE

Yahoo Slurp

Business / Internet Marketing / Yahoo Slurp: Yahoo web crawler. Instructions to Yahoo Slurp can be directed in the robots.txt file. Googlebot statistics can be viewed with web analytics software, on webserver logs, and in Google webmaster tools. MORE

Canonical Tag

Business / Internet Marketing / Canonical Tag: Code used by search engine crawlers/spiders to tell search engines what URL is the original version of your webpage. MORE