As of January, 2012, this site is no longer being updated, due to work and health issues
Search tools covered on this site index a single site, an Intranet or a limited set of sites creating a portal. Web-wide search engines attempt to cover the entire Web, indexing billions of pages.
Google (link-analysis means that the most useful pages are often first)
HotBot (default "and" on multiple terms)
AltaVista (lots of options like link: and url:)
AllTheWeb (useful results using the FAST search engine)
Yahoo, because librarians do rule.
AskJeeves, because people are smarter than machines
Many web pages with useful information are stored in databases and generated as HTML pages only when searched, so they are not indexed by the standard robot spiders. This is known as the Invisible Web or the Deep Web. Some companies are now attempting to provide search access to these databases without making searchers go to the sites themselves. For example, La Scala's travel schedule or the Where the Wild Things Are video. Auction metasearch engines are a form of deep web searching.
Many search tools will index all available URLs, even if they have a question mark in the path. This allows them to combine the deep and surface web and create a comprehensive index on a particular topic. All they need is a set of links to the contents of the database, or a direct connection to a database, like AltaVista and Excalibur RetrievalWare.
The Invisible Web for Educators
The Invisible Web About.com, June 11, 1999 by Chris Sherman - Nice overview of the problems with accessing information in databases.
Direct search Gary Price - Links to search interfaces for databases and other special information resources.
Meta searching involves sending queries to multiple webwide search engines and grouping the results in a useful way. This function can be performed by an engine at a web site, by a local application, or by a site server application.
Dogpile - shows you the top results from various search engines
MetaCrawler - searches financial information and shareware as well as search engines
Web Sites That Help in Your MetaSearch San Francisco Chronicle, October 20, 1998 by Avi Rappoport
Copernic 2000 -Windows application that combines search results in a window.
Sherlock - part of Mac OS, many plug-ins for additional search sites available.
Searching Every Internet Cranny / New tools let you query multiple engines and combine results San Francisco Chronicle, October 20, 1998 by Henry Norr
SearchTools report on adding a meta search engine to a web site or Intranet.