A web spider is a software program or service that can crawl the world wide web. It starts from one or more seed sites/pages then finds and follows links to discover other sites and pages.
Most web spiders are desgined to extract specific information from a page. For example Internet search engines extract the content so the page can be indexed. Price comparrision engines extract product information and prices.