Week 3: Week 3:

MetaSearch EnginesMetaSearch Engines

Click here for Word handoutTom Johnson

Boston University - Dept. of JournalismTom@jtjohnson.com

2 JO807 © J.T.Johnson 2000________________________________

MetaSearch Engines: Definition

You submit keywords in search box Engine transmits search simultaneously to

several individual search engines and their databases of web pages.

Results back from all the search engines queried, usually deleting duplicates

Meta-search engines do not own a database of Web pages; they send your search terms to the databases maintained for other search engines.

3 JO807 © J.T.Johnson 2000________________________________

How do you know if your search terms will "work"?

Search protocol (the way you enter search keywords) is far from standardized

Almost all accept " " as causing a phrase A few accept Boolean AND, OR, and NOT. Fewer accept ( ) to group terms. Some only accept + or -. Some default to OR, some to AND Some take * to truncate. Other stem

automatically

4 JO807 © J.T.Johnson 2000________________________________

Determining the usefulness of any meta-search engine?

Which search engines they send your search terms to (size, content, number of search engines, you ability to choose the search engines you prefer. All of them search subject directories as well as search engines and intermix results from all.)

How they handle your search terms and search syntax (Boolean operators, phrases, and defaults they impose);

How they display results (ranking; aggregated into one list, or with each search engine's results reported separately)

5 JO807 © J.T.Johnson 2000________________________________

Use metasearch, but cautiously Meta-search engines only spend a short time

in each database; often retrieve <10% of any of the results in any of the databases queried. 

Most meta-searchers simply pass your search terms along; if search contains more than one or two words or very complex logic, most of that will be lost. 

Quantity in results does not equal satisfaction. Look for meta-search engines that also send

your terms to selective or odd databases like WebCrawler, Thunderstone, Direct Hit, and WhatUSeek.

6 JO807 © J.T.Johnson 2000________________________________

Types of Search Engines

Search Engines & Meta-Search Engines

Subject Directories Subject Guides Specialized Databases

The Invisible Web Edgar

7 JO807 © J.T.Johnson 2000________________________________

Types of Search Engines: Search Engines & Meta-Search Engines Full-text of selected Web pages Search by keyword, trying to match words in pages No browsing, no subject categories Databases compiled by "spiders" (computer-robot

programs) with minimal human oversight Search-Engine size: from small and specialized to 90+

percent of the indexable Web Meta-Search Engines quickly, superficially search

several individual search engines at once. Return results compiled into a sometimes convenient format.

CAVEAT: Only catch about 10% of search results in any of the search engines they visit.

8 JO807 © J.T.Johnson 2000________________________________

Types of Search Engines Search Engines & Meta-Search Engines

Examples: Alta Vista, Northern Light, Infoseek, Fast

Search Other search engines:

Hotbot, Lycos, Excite Meta-Search Engines:

Metacrawler, Inference Find, Dogpile, Metafind

9 JO807 © J.T.Johnson 2000________________________________

Types of Search Engines:Subject Directories

Characteristics: Hand-selected sites picked by editors, more or less

carefully Organized into hierarchical subject categories Often annotated with descriptions (not in Yahoo!) Browse subject categories or search using broad,

general terms NO full-text of documents. Can search only the subject

categories and descriptions

10 JO807 © J.T.Johnson 2000________________________________

Types of Search Engines:Subject Directories

Examples: Librarians' Index, Infomine, Britannica's Internet

Guide, Yahoo!, Galaxy Other subject directories: Scout Report Signpost, Looksmart, Lycos's A2Z

and Top 5% Built into Infoseek, Excite Most university libraries maintain subject

directories

11 JO807 © J.T.Johnson 2000________________________________

Types of Search Engines:Subject Guides

Characteristics: Web pages of collections of hypertext links on a

subject Compiled by "expert" subject specialists,

agencies, associations, and hobbyists Locate through special guides to guides or

subject directories or sometimes among search engine keyword search results

12 JO807 © J.T.Johnson 2000________________________________

Types of Search Engines:Subject Guides

Characteristics: Web pages of collections of hypertext links on a

subject Compiled by "expert" subject specialists,

agencies, associations, and hobbyists Locate through special guides to guides or

subject directories or sometimes among search engine keyword search results

13 JO807 © J.T.Johnson 2000________________________________

Types of Search Engines:Subject Guides

Examples: Guides to guides:

Argus Clearinghouse WWW Virtual Library

14 JO807 © J.T.Johnson 2000________________________________

Types of Search Engines:Specialized Databases

Characteristics: The Web provides access through a search box into

the contents of a database in a computer somewhere

Can be on any topic, can be trivial, commercial, task-specific, or a rich treasure devoted to your topic

Locate by special guides to databases, in Librarian's Index, Yahoo!, and sometimes by keyword searching general search engines

15 JO807 © J.T.Johnson 2000________________________________

Types of Search Engines:Specialized Databases

Examples: Sites listing lots of Databases

Search.com www.search.com The Invisible Web www.about.com Edgar www.edgar-online.com FECInfo www.fecinfo.com Adobe PDF files searchpdf.adobe.com

16 JO807 © J.T.Johnson 2000________________________________

A good place to start

BEST General Web Page SEARCH ENGINEShttp://www.lib.berkeley.edu/TeachingLib/Guides/Internet/SearchEngines.html

A Graduated Approach in 5 Stepshttp://www.lib.berkeley.edu/Help/search.html

Search Engine Guide http://www.searchengineguide.com 3,600+ search engines in its index. Will take you to search engines that target specific industries or topics, including 67 for "News and Media." This homepage also has a link to its mailing list, a good listserv that helps you stay on top of new developments in the metasearch world.