No matter how carefully you design a search engine, there will always be some searches that fail. Our search log analysis shows that it's often a simple mistake or misunderstanding, so there's no one to blame. Some search engines are better than others at finding useful matches and providing helpful pages when nothing can be found. Top Five Reasons Why Searches Find No Matches 1. Empty searches Amazing numbers of people just click the search button or press Return when the cursor is in the search field. So the search engine gets an empty query, which is usually treated as a search failure. What to do: either chose a search engine which brings up a helpful search page when the search is empty, or make sure that you have wording on your no-matches page to explain what is going on. 2. Wrong Scope (trying to search the whole Web) Whether it's a site, Intranet or small portal, your search engine covers only the topics on your site, not the entire Web. For example, searching for shot put on a site about medieval art is just as useless as searching for marginalia on a sports portal! Despite this, people often see the search field or button, and think they're getting a webwide search engine. If you look in your search logs, you'll see the wide variety of searches which clearly are not directed correctly. What to do: Several search engines provide versions with options to search the entire web, as well as the local site or portal. In any case, the no-matches page should explain the scope of the site and explain what materials the search index covers. 3. Vocabulary Mismatch Searches containing terms which are too specific, too general or just not used. For example, on a medical site, someone searching for doctor may need to look for physician; a horse site may talk about Paso Finos instead of walking horses; and a beginning web design portal may not include any sites that get into details on onblur tags, though they may have some links to JavaScript pages. What to do Some search engines perform linguistic stemming -- they search for several forms of a word instead of just the exact match. For example, a search for run might also find running, runs, ran, or runner; a search for goose might also look for geese. This makes it more likely that a search will find a match, but must be implemented carefully so it doesn't display inappropriate matches at the top of the results list. Note that multilingual sites must be very careful in implementing the correct stemming algorithm for each page's languages, or they may return some very odd results. Adding metadata to pages can help match search words, especially for broader and narrower terms. If you have pages describing DSL and cable modems but never use the word broadband, adding that term to the META Keywords tag content will allow the search engine to find those pages Search engines may allow search administrators to set up synonym lists or thesari to provide appropriate alternate terms. For direct synonyms, such as urticaria as the technical term for hives, it's appropriate to simply add that to the query. For less obvious equivalents, such as red for crimson, and for broader and narrower terms, the search engine should display them and allow the searcher to click on them rather than typing them in. 4. Spelling Mistakes People make mistakes in spelling all the time. For simple typos, log file analysis shows that searchers will re-enter the word correctly. However, they often are unable to remember unfamiliar terms such as diseases, product codes, and names in general. What to do Synonym lists (described above) can help with common misspellings of important words on a site -- the engine can simply translate the bad spelling to the correct version and continue. A spellchecker can provide a list of correctly-spelled words, allowing searchers to switch to the right spelling. Complex Solutions Information Retrieval theorists have come up with some clever solutions to spelling errors, but they can be frustrating to users unless presented properly. Be careful when implementing a search with fuzzy matching: use usability testing and search logs to track whether the results are substantially better than exact matching. Fuzzy matching techniques try to reduce words to their core and then match all forms of the word. For example, searching for serach would properly locate search, but a search for locks might find looks, whether or not that was wanted. Phonetic, sound-alike or "Soundex" matching uses linguistics to search for words which may sound similar. This is particularly useful for names, so a search for licos will find Lycos, but also brings up odd results: a search for fuzzie may match fees or face. 5. Query Requirements Not Met If the search engine automatically searches for all terms or a phrase, or the query includes operators such as + or NOT, there may be no pages which fulfill all the requirements. Examples include brown bear(both terms), "Olympic gold medal" (phrase), claymation +British (required term), MP3 NOT Napster (excluded terms). This can also happen when searching in a particular section or zone of a site for words which are used in different sections of the site. Yahoo and CNet are good examples where people can limit searches to a subsection, but searching for roses in the tropical fish area will never find a match. What to do: Search engines which show the number of pages matched for each term are particularly helpful in this case. The no-matches page can clarify which of the terms caused the problem and provide advice on how to enlarge the search. Other Causes of Search Failure Problems with Query Syntax: some search engines are very picky about how the query must be entered. If a searcher puts a space after an operator such as +, or uses NOT instead of AND NOT, the search may fail. The search engine should provide helpful error messages and instructions in this case, or better yet, be more accepting in its query parsing. For example, if a close parenthesis ")" is missing, the engine can simply add it at the end of the query, with an explanatory message in the results page. Capitalization and Extended Characters: some search engines require exact matches for capital letters and diacritical characters (such as ü, ß, ñ). In that case, searches for pokeman will never find pages with the word Pokéman. If your search engine is strict about these elements, make sure that explain the problem on the no-matches page. Stopwords: to save index space and time, some search engine just don't include common words in their index. This can range from prepositions and conjunctions (such as a, an, the, with, from) to words which are extremely common within the index (baseball and TV on a sports site, for example). Unfortunately, people often search for these terms (think of As You Like It) and are confused when they can't find the pages which contain them. If your search engine must use stopwords, make sure the no-matches page explains what they are and how to search around them. Short Words: some search engines have a lower-size limit for indexing words, so they don't have to store thousands of entries for the word I and other short words. However, if those are required parts of a search, omitting them from the index means that the search nay not find any matches (to be or not to be, for example). Whenever possible, index everything. If your search engine will not index short words, make sure the no-matches page explains how to search around them. Numbers: some search engines don't index numbers, either because they are short or just because they are not words. But people search on them all the time! If your search engine will not index them, make sure the no-matches page explains that numbers can't be searched. Avoiding Future Failure Be sure to track your search logs so you can see why and how your customers are having problems with search. Watch those searches that find 0 matches like a hawk, and do your best to add new synonyms, terms and information that addresses these questions.