EVALUATING USERS EXPLORATORY INFORMATION
SEEKING BEHAVIORS
A study submitted in partial fulfilment of the requirements for the degree of Master
of Science in Information Systems
At
THE UNIVERSITY OF SHEFFIELD
By
MADHUSUTHANAN NITHYANANDHAM
September 2011
i
ABSTRACT
Background The increase in number of internet users searching through the web for their
various information needs has led to increase in number of exploratory search activities due to
which users face the problem of uncertainty of their search goal. Therefore there is need of
search system to support efficient information retrieval by users for their exploratory search
tasks. Previous researchers have identified and developed some systems to support users’
exploratory search but the search system should support and be part of daily search activities of
people.
Aims The aim of this research study was to identify and analyze the exploratory search behavior
of users, evaluate the existing search systems to identify the support provided by those systems
for users exploratory search, analyze users exploratory searching behavior to identify the role of
domain knowledge for exploratory search and to design an alternative interface that supports
better and easier exploratory search.
Methods A survey questionnaire was designed and piloted with two of my friends. The final
survey questionnaire was distributed in-person, online through email and social networking site
Facebook to selected sixteen participants. Experimental search task was conducted followed by
distribution of evaluation questionnaire and interview session. Finally evaluation of designed
alternative interface was carried out followed by interview session.
Results Every internet users have an experience of exploratory search. Most of them are not
aware of exploratory search systems and they use only search engines for their exploratory
search. But neither of the systems supports their exploratory search better. Most users preferred
query suggestions and view-based search and among the techniques to support exploratory
search most of the users preferred tags and faceted interface. The alternative interface was found
useful by users and support better exploratory search.
Conclusions The analysis of results and findings and development of alternative interface based
on users’ feedback and future suggestions will support better and easier exploratory search.
ii
ACKNOWLEDGEMENT
I would like to express my gratitude to my supervisor Prof. Nigel Ford for providing me support,
encouragement and guidance throughout the course of this dissertation. Also I would be grateful
to Dr. Paul Clough who provided me motivation to take up this study.
I would like to thank all participants who participated in various research methods of this study
by dedicating their precious time.
I am thankful to my parents and friends for their constant support and encouragement throughout
my studies.
I dedicate my dissertation to my “Parents, Friends and Teachers”.
iii
TABLE OF CONTENTS
S. No. Contents Page No
1 INTRODUCTION 1
1.1 Introduction 2
1.2 Background and Motivation 3
1.3 Aim and Objectives 4
1.4 Research Questions 5
1.5 Outline 5
2 LITERATURE REVIEW 6
2.1 Introduction 7
2.2 Exploratory Search 8
2.3 Exploratory Search Systems (ESSs) 9
2.3.1 Social Tagging Systems 9
2.3.2 Visualization Techniques 12
2.3.3 Surrogates 14
2.3.4 Faceted Interfaces 15
2.3.5 Semantic Web 20
2.3.6 Relational Browsers 23
2.3.7 Clustering Systems 25
2.4 Exploratory Search Methods 28
2.4.1 Iterative Methods 28
2.4.2 Query by Example 28
2.4.3 Keyword Refining Through Feedback 28
2.4.5 View Based Search 29
2.5 Role of Domain Knowledge for Exploratory Search 29
2.6 Analysis of Literature Review 30
iv
3 METHODOLOGY 32
3.1 Introduction 33
3.2 Research Methods 33
3.2.1 Experimental Research Methods 33
3.2.2 Optimal Based Research Methods 34
3.2.3 Observational Research Methods 35
3.3 Research Approaches 36
3.4 Research Strategies 38
3.5 Methods Used in this Research Study 38
3.6 Procedure 39
3.7 Questionnaire Design 40
3.7.1 Self-completion Questionnaire 40
3.7.2 Question Styles 41
3.7.3 Presentation of Questionnaire 42
3.8 Pilot Study 43
3.9 Sampling 43
3.10 Participants Recruitment 44
3.11 Distribution of Survey Questionnaire 44
3.12 Survey Questionnaire Questions 45
3.13 The Experiment 46
3.13.1 The Exploratory Search Task 48
3.13.2 Evaluation Questionnaire 49
3.13.3 Evaluation Questionnaire Questions 49
3.13.4 Interview 50
3.13.5 Interview Questions 50
3.14 Alternative Interface for Exploratory Search 50
3.14.1 Evaluation of Alternative Interface 51
v
3.15 Data Analysis 51
3.16 Ethical Aspects 51
3.17 Practicalities 52
4 RESULTS AND FINDINGS 53
4.1 Survey Questionnaires 54
4.2 Evaluation Questionnaires 71
4.3 Interview 75
4.4 Role of Domain Knowledge for Exploratory Search 77
4.5 Summary of the Findings 77
4.6 Alternative Interface for Exploratory Search 80
4.6.1 General Layout 80
4.6.2 Search Options 81
4.6.3 Keyword Suggestions 81
4.6.4 Search within Results 81
4.6.5 Advanced Search Options 82
4.7 Evaluation and User Feedback on Alternative Interface 83
4.8 Limitations of the Study 84
5 CONCLUSION 85
5.1 Conclusions 87
5.2 Implications of the Study 90
5.3 Suggestions for Future Work 90
BIBLIOGRAPHY 91
APPENDICES 98
APPENDIX 1 98
APPENDIX 2 100
APPENDIX 3 102
vi
LIST OF FIGURES
Page No
Figure 2.1 Exploratory Search Activities 8
Figure 2.2 Dogear social bookmarking service 11
Figure 2.3 Mr Taggy interface 11
Figure 2.4 TagSphere 13
Figure 2.5 Informedia Storyboard- TRECVID 2006 15
Figure 2.6 mSpace faceted column browser 18
Figure 2.7 SERVICE 18
Figure 2.8 PunchStock 19
Figure 2.9 Flamenco – Hierarchical Facet navigation 19
Figure 2.10 FacetBrowser 20
Figure 2.11 Bletchley Park Text 22
Figure 2.12 LED interface 23
Figure 2.13 Relational Browser 24
Figure 2.14 Relational Browsers ++ Interface 25
Figure 2.15 IGroup 27
Figure 3.1 The Research Wheel 33
Figure 3.2 Google 47
Figure 3.3 Delicious 48
Figure 4.1 Usage of Internet 55
vii
Figure 4.2 Experience of Exploratory Search 56
Figure 4.3 Problems faced by users for their Exploratory Search 57
Figure 4.4 Users awareness of Exploratory Search Systems (ESSs) 59
Figure 4.5 Preference of systems by users for Exploratory Search 60
Figure 4.6 Ability to use Search Engines 62
Figure 4.7 Ability to use Exploratory Search Systems 63
Figure 4.8 Preference of Search Engines by users for Exploratory Search 64
Figure 4.9 Approach to support user’s exploratory search 65
Figure 4.10 Support by Search Engines or ESSs for exploratory Search 67
Figure 4.11 User’s awareness of various techniques to support exploratory search 69
Figure 4.12 Technique to support exploratory search 70
Figure 4.13 Alternative interface: General Layout 81
Figure 4.14 Alternative interface: Search Options 82
Figure 4.15 Alternative interface: Advanced Search Options 83
viii
LIST OF TABLES
Page No
Table 4.1 Usage of Internet 54
Table 4.2 Experience of Exploratory Search 55
Table 4.3 Problems faced by users for their Exploratory Search 57
Table 4.4 Users awareness of Exploratory Search Systems (ESSs) 58
Table 4.5 Preference of systems by users for Exploratory Search 60
Table 4.6 Ability to use Search Engines 61
Table 4.7 Ability to use Exploratory Search Systems 62
Table 4.8 Preference of Search Engines by users for Exploratory Search 63
Table 4.9 Approach to support user’s exploratory search 65
Table 4.10 Support by Search Engines or ESSs for exploratory Search 67
Table 4.11 User’s awareness of various techniques to support exploratory search 68
Table 4.12 Technique to support exploratory search 70
1
CHAPTER 1
INTRODUCTION
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
2
1.1. Introduction
The development of World Wide Web and internet has led to increase in number of people
searching web for different daily purposes and database storage and retrieval ranges from
database to search engines. There is search for information needs by professionals in their
workplace to develop knowledge for their work practice to common people for their various
daily activities. As a result, problem of effective retrieval of information is a concern for various
research studies and various search systems are developed during these days to satisfy people
information needs.
According to Internet World Stats (2010) by Miniwatts Marketing Group there are
1,966,514,816 (estimated) internet users in the world which is 28.7% of overall world
population. Every internet users search the web through various search systems to gather their
desired information. Some search activities will have undefined goal and searchers does not have
proper domain knowledge about search subject in which they does not aim to obtain specific
information but aims to gather related information about a subject from various sources to
develop their knowledge. This form of searching behavior leads to exploratory search. Examples
of exploratory searches are finding information about foreign trip, users finding particular
information in website for first time and browsing best book seller lists to buy a book. Also
domain knowledge is important for searchers to find exact information they need.
Traditional search systems require users to have a definite search goal and to submit a
definite query relevant to their search to initiate a search process and to get desired results. These
types of search activities are not possible for exploratory search which arises when users are
unfamiliar which keywords to use for search, unaware how to find the information and
unfamiliar where to find the information. Mostly the submitted keywords may not produce
expected results. Also this will be time consuming as searchers need to search through different
ways and when they search in particular website they need to learn about that website better
before searching process. There is broader scope of research in this due to ability of some users
to perform only keyword search and when the user is unaware about which keyword to use for
search.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
3
1.2. Background and Motivation
Exploratory searchers will have uncertainty of their search goal as they aim to collect
broad information about subject. Exploratory searchers submit tentative query to collect any
general information about subject and they explore search results for their next step by refining
their search terms and by modifying their search goals. They need to collect, analyze and
integrate information from various sources to get the information they need. Exploratory
searchers are unsure about the quality of information they obtain from search systems as there is
need for them to assure and interpret documents returned by search systems for their search
query. Exploratory search is iterative process that requires users to search through series of
queries and it depends on the ability of user to analyze and navigate the search results to number
of new and previously visited pages to obtain the desired results (Bates, 1989). But these types of
search activities such as refining queries for iterative search process by users, processing the
search results by analyzing and examining the documents returned by search systems is not
possible by exploratory searchers as they lack knowledge about their search subject and have an
ill defined search goal.
Researchers have identified and developed some search systems and techniques like social
tagging systems, exploratory search systems, visualization technique, exploration map, faceted
interface which will help exploratory search. Each technique has advantages and limitations.
Also there is need for search systems to support effective retrieval of desired information by
users for their exploratory search task. Therefore there is necessary for development of new
systems or technique that will support better exploratory search and that satisfy exploratory
searchers.
As a result, this project aims to study the query methods, search options and presentation
and organization of results provided by existing search systems and to identify strengths and
improvements needed in those systems and design an effective interface system to satisfy users
exploratory search needs. The interface will be designed based on the users’ preference of search
systems, evaluation of existing search systems by users for their exploratory search task and
review of existing literature on search systems to support exploratory search.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
4
1.3. Aim and Objectives
The aim of this dissertation is to analyze the users’ exploratory information seeking
behavior, evaluate existing search systems and the support provided by those systems to user’s
exploratory search task to identify whether it satisfies users information exploratory search needs
and by identifying possible improvements to design an alternative interface that supports better
and easier exploratory search even for common users. Therefore during the literature review
process we review at all existing search engines and exploratory search systems and the proposal
by some researchers and evaluate existing search systems using survey questionnaires,
experimental evaluation and interviews. We try to identify the benefits of each proposed system
and also limitations of those systems. Therefore by this research we try to recognize the best
technique that will support exploratory search by identifying changes and improvements in
existing search systems and propose and design an interface to support better and easier
exploratory search. The alternative interface that will be designed based on existing literature and
based on analysis of data collected from users will be evaluated by users to provide feedback on
the support provided by alternative interface for their exploratory search tasks. Also we try to
identify the role of domain knowledge in exploratory information seeking. There were many
researches going on to identify the role of domain expertise in the information seeking behavior
which is the level of education about domain of searching process. The objective of this
dissertation is to identify and analyze users’ experience of exploratory search, their awareness of
present search engines and exploratory search systems, their experience with those systems and
their preference of those search systems, techniques and approaches for their exploratory search
task. Also this study evaluate the existing search systems to identify the support provided by
those systems to users exploratory search and to analyze users exploratory searching behavior in
search engine and ESS to identify the role of domain knowledge for exploratory search and
design an alternative interface that will support better exploratory search task and to evaluate the
support provided by alternative interface design for users exploratory search.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
5
1.4. Research Questions
1. What will be the searching behavior of users for exploratory search task?
2. To what extent the existing search systems/tools support exploratory search?
3. What is the role of domain knowledge in exploratory search, how did it influence the
search and the information collected?
4. Would the proposed interface technology make change and support exploratory search
behavior?
1.5. Outline
Chapter 2 describes the background of exploratory search and its challenges. It also describes the
previous studies of various search engines and exploratory search systems, its merits and
demerits along with approaches and techniques.
Chapter 3 describes the methodology used in this research study along with justification of
choosing adopted methodology. It also describes the data collection methods, data analysis
procedures, ethical issues and practicalities associated with this research study.
Chapter 4 describes the results and findings from data collected through various adopted data
collection methods. The results and findings were analyzed in the context of existing literature
and findings were summarized. It also describes the alternative interface design and summaries
users’ feedback on alternative interface design.
Chapter 5 presents the conclusion of this research study. It also explains the limitations of this
study and suggestions for future work with respect to this research study.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
6
CHAPTER 2
LITERATURE
REVIEW
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
7
2.1. Introduction
Most of the daily activities of people involve searching for information through the
internet. The internet provides communication and exchange of information between people.
Some of the issues rose due to growth of internet are information overload, information
unavailability, navigation problems and diversity of different users. The user’s web search
session involves submitting a query, selecting suitable link from returned results and browsing
the link page, navigating the web page and modifying their query if they are unsatisfied with the
acquired information. Some of the search of the people will have clear goal to achieve but most
of them will have unclear goal and unaware about the exact location to find the information and
unsure about how to find the exact information. The most used keyword-based search is offered
by many search engines such as Google, Bing etc. in which users query through set of keywords
for their search activity along with advanced search option. But the keyword-based search and
advanced search option is difficult and complex due to difficulties in effective query formulation,
modification using suitable keywords to obtain desired result and unsuitable to explore
information space. JcsCholtes (2010) differentiated search in to two main types – Focalized and
Exploratory. According to him Focalized search also known as web-search or portal search, in
which users will have well defined goal for their search behavior and looks for particular best
result. Focalized search are suitable only for personal search and does not help in investigation of
a topic as focalized search lack navigation of relevant results. In Exploratory search, users will
be unfamiliar with the goal, unsure about goal and how to achieve the goal.
Marchionini (2006) explained the three kinds of search of search activities such as
Lookup, Learn and Investigate as shown in Figure 2.1 and outlined the process carried out during
each activity. Lookup activity is supported by most of the search engines in which users are
aware of their search goal and they give specified query to yield selected discrete results.
Searching through search engines displays results ranked in any order but exploratory search
which is less directed requires search results to be well organized to obtain better information
about search topic. Cove and Walsh (1988) classified browsing in to search browsing in which
browsing is directed and users have clear goal to achieve, general purpose browsing in which
browsing is based on specific topic or area of interest and serendipity browsing in which
browsing is a unstructured activity. Exploratory browsing involves general purpose and
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
8
serendipity browsing in which users will have specific area of interest but does not have specific
goal to achieve. Marchionini (2006) explained exploratory search as search to learn and
investigate in which users browse and investigate information in multiple search sessions to
acquire broader information which cannot be satisfied by the relevant page result that will be
acquired through focalized search. Exploratory search combines querying and browsing that help
will help users to investigate particular topic by navigation and discovery of relevant results.
Figure 2.1 – Exploratory Search Activities (Marchionini, 2006)
2.2. Exploratory Search
Exploratory search grown from the field of Information Retrieval and Information Seeking
and covers various activities such as investigating, evaluating, comparing and synthesizing.
Exploratory search are carried out in everyday life by exploring new information space to gather
the knowledge about a domain which help them make a decision. Exploratory search task
requires using more queries to search, remembering and revisiting previously visited web pages,
visit new domain areas thus requires more workload to be carried out for the task (Park et al.,
2008). Thus exploratory search requires people to submit a tentative query to navigate a
collection of related information, to explore the obtained information to obtain cues about their
next step that helps to perform multiple query iterations to obtain a broader class of information.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
9
2.3. Exploratory Search Systems (ESSs)
Exploratory Search Systems (ESSs) such as information visualization systems, document
clustering and browsing systems, and intelligent content summarization systems support decision
making through and maximize information gain through information foraging (White et al.,
2008). ESSs help users to make decisions about which hyperlinks to follow for the next step of
their exploratory search process and help them to refine their searching for the shortest path to
achieve the solution for the search topic. According to White et al. (2006), effectiveness of
exploratory search systems should be evaluated based on measurement of interaction behavior,
cognitive load and learning that provided by the systems.
2.3.1. Social Tagging Systems
Social bookmarking or tagging systems support simple search where the users are aware of
their search goal and domain to exploratory search where users are unaware of their search goal
and domain. Tags are associated with every data objects such as photos, URLs, blogs, articles,
audios and videos that help users to organize, access the data objects and retrieve them
effectively when shared between different users. In simple or known-item search users search
through specific queries and analyze the search results to find appropriate bookmarks to gain the
specific information needed to them. Millen et al. (2007) explained the support provided by
social bookmarking applications for known-item search in which users browse through
bookmarks or query through tags for personal bookmark browsing and the support provided by
social bookmarking applications for exploratory search in which users browse through
bookmarks according to time and bookmarks which is popular. Social tagging systems are most
desirable sometimes because social tags from other user’s acts as navigation cues that help in
information acquisition and facilitate exploratory search (Millen, et al., 2007). Social tagging
systems provide better social information environment in which tags also interpreted as
folksonomies provides information cues from others user’s that helps to learn and acquire
knowledge to find the correct information. Customized tags help individual for future research
and social tags helps other users to search information with similar interest (Kammerer, et al.,
2009). The information contained in the tags helps the users to find the relevant concepts and
resources and the inferred topics helps to find the related contents. Thus social tagging systems
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
10
help in topic inference process and topic extraction process thus acts as semantic imitation model
as explained by Fu (2008). Therefore various features such as creating bookmarks and sharing it
with others, use of tags or keywords for bookmarks that helps users to identify the bookmark and
classify the bookmarks in to collections and social use of system are supported by number of
social bookmarking systems. The problem in social tagging systems is that social tags generate
noise as users have freedom to choose any keyword as tag (Chi and Nairn, 2009). The users may
choose wrong keywords when tagging, some words used as keywords to tag may have different
meaning and tags used in different languages are the problems associated with tagging data
items.
Kerr et al. (2006) proposed a social bookmarking system for large scale organizations
called Dogear social bookmarking service shown in Figure 2.2 that offers number of distinct
characteristics or features such communication between colleagues of organization, bookmarking
intranet resources, slide control to view tag cloud that contain histories of tag and tag index,
navigation of individual people bookmark collection through people links, navigation of
bookmark for which tags associated with other tags through tag clusters, bookmark search
through query search in search box or through drop down search option of people name, tag or
bookmark and community browsing mechanism. The web search tools such as DeeperWeb and
Search Cloudlet helps the users in exploratory search by suggesting keywords that helps the
users to refine their query with the new terms related to the query and also the tools which are
available as plug-ins can be integrated with search engines such as Google that related the results
of users search query from search engines with tag clouds so that user can refine their query with
terms in tag cloud. Kammerer et al. (2009) suggested user interface for tag browser called Mr
Taggy shown in Figure 2.3 that identifies relationship between tags and suggest new tags and
concepts by browsing through documents. It also helps the user to provide Relative tag feedback
and related search feedback that helps to identify related concepts and URL for keyword used to
search. Krause et al. (2008) analyzed the use of queries in search engines and tags in social
tagging systems by people for exploratory search and found that tags are useful than search
results produced by search engines. Also they argued that social tagging systems provide an
information seeking environment and help in effective exploratory search. However there is no
definitive evidence to show that social tagging systems are better than search engines and
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
11
important role of domain knowledge when searching through social tagging systems for
exploratory search.
Figure 2.2 - Dogear social bookmarking service (Kerr et al., 2006)
Figure 2.3 – Mr Taggy interface (Wilson et al., 2010)
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
12
2.3.2. Visualization Techniques
Visualization of web space helps to classify the documents and present the results from
web search in structured format to help determine the relevance of different pages and allows
users to select desired pages. Information Visualization is the process of transforming three
different types of data according to Stevens (1946): nominal data which helps to identify things,
Ordinal data which are ordered data and Quantitative data which helps to compute ratio or
interval between data values in to graphical attributes which helps to recognize patterns and
reduces the cognitive workload. Visualization in web applications offers navigation cues that
help users to explore interesting items and make correct decision based on the interest of others
and based on the attention of most other users (Indratmo, 2010). As explained by Mercun and
Zumer (2009) Visualization helps in exploratory information seeking and used in many types of
search systems such as Library systems, Eyeplorer, Liveplasma, Tuneglue, Musicovery and
Touchgraph but most of those lack information needed to represent visual data, textual data and
require proper architecture and interaction design that will help in proper understanding by users
of system.
Digital Library systems will contain vast amount of information items and therefore
retrieval of information item from digital library system will be time consuming unless users
have proper domain knowledge about the item they are looking for. Relational visualization
technique used for HighWire Digital Library of Stanford University provided with Topic maps
of nodes and edges with nodes represents topics structured according to search query and edges
represent relationships between different topics (nodes). Seifert and Kruppa (2010) developed a
system called Digital Library Assistant (DILIA) using Relational Visualization technique in
which results of user search query is visualized as topics as round blobs labeled with topic names
with number of documents present under each topic. The size of blobs represents the number of
documents and edge between topics blobs represents the documents shared between topics with
thickness of edge represents the number of documents shared between topics. Visualization
technique support exploratory search through graphical display of items and queries in document
space and depicts the relationships among data that helps users in easy navigation through
document space to acquire information they require. Dork et al. (2008) proposed a system called
VisGets for coordinated visualizations that help to visually filter the data from more than one
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
13
dimension such as time, location and topic that cannot be expressed by textual data and construct
dynamic queries to explore large information space. Aurnhammer et al. (2006) developed a
system called TagSphere shown in Figure 2.4 for content-based image retrieval and visualization
of entire search results corresponding to user’s selection of image from examples. TagSphere
combines tagging along with visualization in which tags associated with images are visualized
using different circles that helps users to retrieve images easily. Also users can perform search
using tag where the corresponding image results are displayed in suggestion display and images
selected by users are added to user collection area.
Figure 2.4 - TagSphere (Aurnhammer et al., 2006)
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
14
2.3.3. Surrogates
Document surrogates are representations about contents of information items which helps
to review the information items quickly which reduces time to process documents. Document
surrogates such as titles, abstracts, summaries, storyboards and sections helps searchers in
making decision about their search activity and thus aid in exploratory search by providing cues
about next step during search. According to Marchionini (1997), there are two types of
surrogates such as descriptive surrogates represent the attributes of the document which helps to
sort the documents and semantic surrogates represent the contents of documents in the form of
thumbnails. Surrogates helps in examination of search results thus helps in faster lookup and also
helps to provide detailed overview of information objects thus helps in easy information
extraction.
The surrogates of documents such as titles are fixed representations of document and
surrogates such as summaries are dynamic representations of documents. Summaries are used for
text retrieval, music videos and media file retrieval and also used for web retrieval. The
storyboard surrogate example of which is shown in Figure 2.5 is used for representation of video
segment in time order that maintains traditional flow. Ding et al. (1999) studied the use of video
surrogates and found that combined surrogates such visual images of key-frames and verbal
information of keywords and phrases provide better surrogate for videos that helps in better
sense-making and information integration of video objects. The interface in Open Video Digital
Library (www.openvideo.org) provides number of surrogates such as storyboard, fast forward
and excerpt for visual representation and textual representation for the video collections that
helps the users to effectively explore the videos for various purposes.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
15
Figure 2.5 - Informedia Storyboard- TRECVID 2006 (Christel, 2008)
2.3.4. Faceted Interfaces
The user interface plays a vital role as an exploratory search tool. The user interface is a
Human Computer Interaction tool which includes both software and hardware that helps efficient
interaction between users and computers. User interface is effective tool because of graphical
interface which will make searching process much easier and adopt more browsing and display
options. Faceted search interface is used for number of exploratory search process due to broad
category of search options and results produced by faceted search. Faceted technique helps to
reduce and provide alternative for query reformulation that provide flexible information seeking
strategies which influence searcher tactics and also helps searcher with unclear domain
knowledge to refine their search for efficient information seeking. Faceted interface is useful for
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
16
exploratory search in the way it helps users without prior knowledge of the item to browse and
retrieve information about item by providing number of alternative choices to browse through
number of possibilities and select items based on their constraints and preferences.
Wilson et al. (2008) explained two main approaches of faceted browsing such as
directional column-faceted browsers and non-directional or traditional faceted browsers. In
directional column-faceted browsers, facets are presented as horizontal so that selection of items
by users in any facets filters the facets to right of the facet. In non-directional or traditional
browsers, selection of items in any facet filters the items in subsequent facets to show items
relevant to selected item. mSpace shown in Figure 2.6 is an interaction model explained by
Schraefel et al. (2006) is a directional column- faceted interface in which facets are represented
by columns from left to right and consists of features such as backward highlighting that helps to
know the structure of collections in facets, order, add or remove the facets in collection, dynamic
info view to know the results of the selection of columns and helps to compare the items in the
collection by change of selection. The log analysis of mSpace by Wilson and Schraefel (2008)
showed that easy to use for complicated queries along with basic keyword query search and
produced subjective measures. There are also web search interfaces systems that help to classify
unknown web collections in to facet categories such as Search Result Visualization and
Interactive Categorized Exploration (SERVICE) shown in figure 2.7, Dyna-Cat, Northern Light,
Exalead and PunchStock shown in Figure 2.8 that automatically generate facets for web search
results. Flamenco system proposed by Hearst et al. (2003) shown in figure 2.9 consists of
hierarchical faceted search interface and provided with menu choice for navigation searching of
collection of art, architecture and tobacco documents. Flamenco interface provide category
overview of collections by providing facets and helps the users to know about the number of
documents present under each category. Hearst et al. (2003) conducted usability test for
Flamenco system and evaluated the system based on subjective measures for exploratory tasks in
which the Flamenco system found more useful for users to retrieve images for structured task
and produced high subjective measures. The other faceted search systems that are similar to
Flamenco are Epicurious, a commercial recipe site developed by a company called Endeca and
Exhibit developed by Huynh et al. (2007). The drawback in these types of faceted interfaces is
that they do not help to visualize or identify relationships between different facet categories. To
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
17
rectify this problem, Villa et al. (2008) developed a faceted interface called FacetedBrowser
shown in Figure 2.10 that helps in retrieval of multimedia objects and in performing complex
search tasks in which multiple searches are split in to sub-searches all of which can be visually
seen at same time through different facets. FacetedBrowser supports multiple searches to be
performed in parallel through multiple paths by identifying facets during sub-searches thus
providing relationship between facets. Shneiderman and Kules (2008) studied the faceted search
interface using large-scale log analysis that helps to determine use of interface elements by
searchers and comparative user studies to determine the use of interface by searchers and proved
that faceted search interface helps the searchers in better information access and retrieval. They
also shown that searchers those use faceted interface have broad category of search results which
they can access easily than those who doesn’t use faceted interface to search.
The interface in Open Video Digital Library (www.openvideo.org) provides number of
alternative textual and visual browsing options to search the visual segments in the collection and
provides users with more detailed search results. Also it helps the users to download the video in
their desired format (Marchionini, 2006). The interface used in NCSU OPAC that supports
exploratory keyword search consists of three facets such as subject, region and time period to
search all catalog fields in NCSU system (Kules et al., 2009). According to research of Park et
al. (2008), a new interaction tool that provides interface between user and search engine called
SketchBrain helps in information seeking activity by conceptualizing the trails in workspace,
perform implicit operations and helps users to navigate next informative page. The disadvantages
of faceted interface are the faceted classification of information does not provide information
according to ranking or popularity and construction of faceted interface requires predefined
schema and stable structure.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
18
Figure 2.6 - mSpace faceted column browser (Schraefel et al., 2006)
Figure 2.7 – SERVICE (Wilson et al., 2010)
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
19
Figure 2.8 – PunchStock (Wilson et al., 2010)
Figure 2.9- Flamenco – Hierarchical Facet navigation (Hearst, 2006)
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
20
Figure 2.10 - FacetBrowser (Villa et al., 2008)
2.3.5. Semantic Web
The semantic web provides detailed and additional information in the form of metadata
about resources and their relations present in the web. Semantics may be lightweight semantics
which involves sharing information between users through web and strong semantics which
involves sharing information through ontologies. A semantic web search engines consists of a
centralized data repository that contains metadata about documents and resources that are
interlinked and distributed throughout the web. Semantic web technologies have a better
potential to support exploratory search as semantic web search tools provide search results in
organized structures and help to specify new search queries for users to select for their
navigation of content. Semantic web technology provides better search and exploration of
collection of different museums as museums contains semantically rich collections of metadata.
Hyvonen et al. (2004) developed a system MuseumFinland Portal for publishing artefacts from
museum collections of three different museums of Finland on the semantic web. The main goal
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
21
of this semantic web portal is to provide global view to museum collections, provide content-
based search for browsing the interested content and to create national publication channel for
publishing semantically enriched contents of museums on the semantic web.
Mulholland et al. (2008) developed exploratory semantic search applications called
Bletchley Park Text (BPT) shown in Figure 2.11 for museum called Bletchley Park museum in
United Kingdom. BPT consists of newspaper style homepage that provides links to stories that
are generated by matching metadata concepts to topic selected by users and composite pages that
contains organized collection of stories. The metadata in semantic web was also utilized by
Hemel et al. (2003) who developed a prototype called OmniPaper to enhance searching in online
newspaper archives using Topic Maps. OmniPaper makes use of both navigation and querying as
it supports two types of navigation and querying: hierarchical and relational navigation, simple
and advanced querying in which enhanced relational navigation and querying is supported by
Topic Maps of semantically rich keywords or concepts related to user query produced using
knowledge map. Using hierarchical traditional navigation users can navigate through categories
of news subjects and relational navigation helps to navigate related news subjects produced using
knowledge map. Simple querying produces results of news subjects corresponding to user search
query whereas advances querying helps to include constraints in their querying which returns
filtered results of news subjects.
mSpace which is interaction model and directional column-faceted interface is based on
semantic web technology that used metadata of content to construct terms that helps to explore
the content. Mirizzi and Noia (2010) proposed a web search tool called Lookup Discover
Explorer (LED) shown in Figure 2.12 that helps in exploratory search and enhances semantic
web technology Linked Data. The LED takes account of the semantics of tags in the tag cloud
that are related to search query used by searchers and thus helps in exploratory search by
generating tag cloud with tags or keywords that are semantically related to keywords used by
searchers for their exploratory search, suggest keywords for search query used by searchers and
helps to refine search results from search engines by refining the query used by searchers by
semantically relating the keywords to keywords in a tag cloud. Semantic versions of Wikipedia
that help to exploit Wikipedia include systems such as DBPedia which helps to visualize
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
22
semantic data and OntoWiki which provides lightweight semantics and full text semantic search.
Berners-Lee et al. (2006) proposed a generic data browser similar to DBPedia called Tabulator
project that helps to explore Linked data and visualize Linked data by table, map, calendar and
timeline. The main challenge in the semantic web is the construction of semantic queries by
users as this requires knowledge of query languages and domain knowledge related to respective
URIs.
Figure 2.11 - Bletchley Park Text (Mulholland et al., 2008)
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
23
Figure 2.12 - LED interface (Mirizzi and Noia, 2010)
2.3.6. Relational Browsers
An interactive overview that depicts the structure and content of website is necessary for
large scale websites such as government websites, corporate websites and other portals which
can help users in easy navigation of website. Such interactive overview is provided by generic
interface called Relational Browser that helps to understand the structure of collections in the
web space and provides effective searching by examining relationship between attributes in the
web space. The first prototype of Relational Browser adopted mouse over control mechanism
and used to identify relationships between topic attributes which is used for Fedstats portal
(Marchionini, 2000) shown in Figure 2.13 and the interface developed from the prototype after
usability testing is used for Library of Congress National Digital Library. Marchionini (2006)
suggested Relational Browser for Open Video Digital Library that provides relationship using
mouse brushing between facets such as topic, time, space or data format and their attributes.
Relational Bowser interface has unique features such as displaying results by intersection of
documents based on user’s selection and graphical representation on each facet to show the
number of documents that can be found by users by clicking the facet.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
24
Zhang and Marchionini (2005) developed the Relational Browser++ shown in Figure 2.14
overcoming the limitations in earlier relational browser in which they provide visual overview of
information space with help of multiple facets with facet values represented by graphic bars,
helps partition of information items and to identify relationship between them and provides
dynamic filtering option for search results by which users can filter the search results to obtain
desired results by searching with related keywords. Also Relational Browser++ provides
overview of updated results using result set panel that helps users to explore the information item
for their needs and helps to locate information items easily by matching search string patterns
with information items anywhere in the information space. Capra and Marchionini (2008)
proposed Relational Browser 07 that consist number of features to unique features compared to
previous versions of Relational Browser such as multiple view and facet cloud to display facets,
grid or list view to display search results, display of search query at the top of the screen along
with history of facets and search queries, search through keyword and search within results.
Figure 2.13 - Relational Browser (Marchionini, 2000)
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
25
Figure 2.14 - Relational Browser ++ Interface (Zhang and Marchionini, 2005)
2.3.7. Clustering Systems
Clustering involves grouping search results into groups with specific topic so that users
can choose topic of interest instead of refining their query for their next step. The search results
produced by conventional search engines are organized according to their ranking based on user
query. The users have to navigate the every irrelevant item to find the desired information among
search results returned by conventional search engines which is list according to some ranking
relevance to user query and contains list of mixed topics. Clustering systems organize the results
in to cluster groups and helps to summarize the cluster names to the users which help users in
easy navigation and retrieval of documents from the desired clusters. Thus Clustering systems
help to avoid navigation through multiple search results produced by conventional search
engines and it acts as a complementary to those by providing entire search result lists as a single
view of clustered results which helps users even without proper domain knowledge about search
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
26
topic to acquire better information by following specific cluster relevant to their search query.
Document clustering systems cluster documents in advance in to hierarchical tree structure and
return results that best matches the user search query from the cluster. But clustering engines
help to cluster search results from conventional search engines thus clustering engines should
require efficient algorithm and design to acquire and process search results and efficient
graphical user interface to display the relationship between clusters.
Hwang et al. (2011) proposed a clustering system to cluster documents based on explicit or
implicit feedback from users that helps to identify the similarity between documents that helps to
avoid problem faced by traditional clustering systems to extract contents of documents and
cluster the documents by finding similarity between them. The image search engine called
IGroup shown in Figure 2.15 proposed by Wang et al. (2007) overcomes the limitation of
different style image search results produced by other image search engines as IGroup groups the
image search results in to clusters with name of cluster groups related to query used to search by
users. The image search through IGroup displays general view results as clusters at the left side
of the interface in the navigation panel with the cluster name at the top of each cluster group and
each cluster displays preview of images present under each cluster which can be expanded to
cluster view results of images under each cluster. The other clustering systems are Lexxe which
uses linguistics approach to extract the meaning of search query and generate results, Biznetic
and Accumo which presents search results in a tree view of clusters and meta-search engines
such as Clusty, Qksearch and Polymeta that collects documents from various systems and
organizes in to clusters. The problem with clustering is to cluster documents in response to user
query and to identify suitable label for cluster of documents.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
27
Figure 2.15 - IGroup (Wang et al., 2007)
As pointed out by Rasmussen (2008), there are many tools developed for exploratory
search and evaluated based on some metrics and user generated tasks but the search system
should be developed such that it should not support exploratory search only on the specific set of
task or user- generated task but it should also support every real-world search activity and the
system should be part of users daily information seeking activity. According to White et al.
(2006), ESSs should be evaluated based on metrics and methodologies. Metrics defines the
evaluation of ESSs during the searching process and after the search process which are the
results produced by ESSs. The searching process involves interaction, behavior of user’s and
decision made by users during search process. Methodologies define the evaluation of ESSs
based on experiments such as interviews, questionnaires and tasks. The users should have
acquired and learnt more information of their search topic. Spink (2002) explained the evaluation
of exploratory search systems using system and task centered approaches. A system approach
deals with evaluating the system based on standard function such as testing precision and recall
capabilities of system to improve document retrieval corresponding to search query. A task
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
28
centered approach focus on the interaction of users with the system on how well the users
perform while using the system. Task centered approach involves assigning tasks to be
performed by users and takes account of effectiveness, efficiency and satisfaction of the users
while performing task in the system. The tasks to evaluate ESSs should be based on the domain
knowledge of users involved in the task and the interviews and questionnaires should be
conducted with at-most care to collect the information needed from the users.
2.4. Exploratory Search Methods
2.4.1. Iterative Methods
Each iteration refines the search query and reduces the size of domain. Also keyword used
to search can be refined through feedback from users after each iteration which can produce
quality queries that can be used to search. But both of these iterative methods will be time
consuming as process need to carried for more iterations until desired result is achieved.
2.4.2. Query by Example
In Query by example, users construct query through navigation by clicking on suitable
example documents related to their information needs. Query by example is a content-based
search that helps to order results based on the examples related to the results. Query by example
are supported in system developed by Aurnhammer et al. (2006) called TagSphere that helps in
content-based image retrieval and visualization of search results corresponding to tags associated
with selected example image by user. Another image search engine IGroup developed by Wang
et al. (2007), also supports query by example search in which the search results of keyword
search performed by users are grouped into clusters related to the query and users can refine their
query by selecting images from the clusters. The drawback of using query by example for image
search engine is that some users may unable to find suitable example image required for their
search.
2.4.3. Keyword Refining through Feedback
User feedback can help to refine and improve the quality of search query. Moussa et al.
(2006) conducted a research to study the potential of user feedback for refining queries in image
retrieval system and found that system utilizing user feedback perform faster than normal system
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
29
and avoids the need to reinitiate search if users are not satisfied with their search results. Social
tagging systems help the users to refine their search based on the feedback from other users
suggested in the form of tags that provide information cues to users. Kammerer et al. (2006)
utilized user feedback to refine the search query and search results for their tag search browser
Mr Taggy in which user feedback helps to refine search results and search query by including or
excluding tags corresponding to search query.
2.4.4. View Based Search
In view-based search, users construct query through navigation of search results and
following path suitable to their information need among many available options. Also users can
refine their query by navigating through suitable path among many successive options in
information space. View-based search is supported by faceted browsers which help the users to
choose facets suitable for their information need with restrictions among multiple facets
available and it is used for applications such as online shopping and job searching.
2.5. Role of Domain knowledge for exploratory search
Domain knowledge plays an important role in information seeking and those with high
levels of expertise in a domain engage in efficient searching and acquire better information. The
person with good domain knowledge will tend to select the best links that will lead his or her
target, input short and optimal search queries and also spend less time searching (Duggan and
Payne’s, 2008). White et al. (2009) investigated the role of domain expertise on search behavior
and found that the domain experts use domain specific vocabularies and own terms in their
search queries and are more successful in their search than non-experts. In case of social tagging
systems, domain experts can perform better than domain novices in assigning relevant tags to
every documents that will be easy for other users to predict the documents based on tags and also
in extracting related contents from documents based on tags. As exploratory search involves
more learning and investigating process, the person with good domain knowledge can perform
better exploratory search and acquire information efficiently.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
30
2.6. Analysis of Literature Review
Exploratory search arises when users searching for information through the
internet have unclear goal and when users are unaware about the exact location to
find the information and unsure about how to find the exact information.
Exploratory search combines both learn and investigate as explained by
Marchionini (2006) and involves general purpose and serendipity browsing as
classified by Cove and Walsh.
According to White et al. (2008), various exploratory search systems such as
information visualization systems, document clustering and browsing systems,
and intelligent content summarization systems are available that supports decision
making.
Social tagging or bookmarking systems supports exploratory search in the form of
tags or bookmarks also interpreted as folksonomies from other users that act as
navigation cues that help users learn and acquire knowledge to aid better
information acquisition and thus facilitate exploratory search.
Visualization systems helps in graphical display of items and results from web
search that help users in decision making and to explore the information space in
desired path to select interesting items.
Document surrogates supports exploratory search by providing cues in the form
of representations about contents of information items to review them quickly
which helps in decision making and reduces time to process documents.
Faceted interface provides view-based search with more browsing, search and
display options to make searching easier and also helps in query formulation to
help even the users with unclear domain knowledge to refine their search query
for better information acquisition.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
31
The semantic web has potential to support exploratory search as it contains
metadata about documents and resources to organize search results and help to
specify new search queries for users to select for their navigation of content.
Relational browser is a generic interface that helps to understand the structure of
collections in the web space and provides effective searching by examining
relationship between attributes in the web space which helps users in easy
navigation of website.
Clustering systems provides search results as single view of clustered group
results that help users to select the desired cluster group topic relevant to their
search query and it also help users even without proper domain knowledge about
search topic in easy navigation and retrieve documents from the desired clusters.
According to White et al. (2006), ESSs should be evaluated based on metrics and
methodologies and Spink (2002) explained the evaluation of ESSs using system
and task centered approaches.
The different types of exploratory search methods are iterative methods, query by
example, keyword refining through feedback and view based search.
As exploratory search involves more learning and investigating process, domain
knowledge plays an important role in exploratory search tasks and users with
high level of expertise engage in efficient searching and acquire better
information than novices.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
32
CHAPTER 3
METHODOLODY
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
33
3.1. Introduction
Research involves finding solution to scientific and social problems by analysis of
information collected through various methods and procedures called research methods.
Research methods involves theoretical procedures, experiments, observation, reasoning and
statistical analysis to collect information on particular topic to find solution to problem. The
research wheel shown in Figure 3.1 describes the different phases of research process.
Figure 3.1 - The Research Wheel (Rudestam, 2007)
3.2. Research Methods
According to Shuttleworth (2008), there are three basic research methods as follows
Experimental Research Methods
Opinion Based Research Methods
Observational Research Methods
3.2.1. Experimental Research Methods
Experimental research methods involve conducting laboratory based experiments to
evaluate or test the performance or usability of application or hardware. Experimental research
methods involves testing of research hypothesis using experimental design methods such as true
experiments, laboratory experiments, quasi-experiments to study the cause and effect of design
or situation corresponding to hypothesis (Trochim, 2006). It also aims at providing solution to
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
34
research problem and generation of theories based on manipulation of experimental research
methods. True experiment involves methods such as pre-post test design, random assignment of
participants to research study and evaluation using experimental group. Quasi-experiments
involves naturally occurring treatment groups and control groups that does not set up or assigned
for research which differs it from true experiment that involves assigning participants to
treatment groups or experimental groups.
3.2.2. Opinion Based Research Methods
Opinion based research methods involves collection of data through questionnaire or
survey, interview or focus groups. Survey involves researchers to gather information by sending
series of research questions to subjects to respond through which more information can be
collected within short period of time. Survey is of two basic types: cross-sectional survey and
longitudinal survey. Cross-sectional survey involves collecting information at a specified time
that may help to gather data about particular factor or compare and identify relationship between
two factors. Longitudinal survey involves collecting information over a period of time to make a
detailed analysis about a factor for that period of time. Survey can be conducted online through
email or through telephone. Telephone survey will be expensive to conduct than email survey
but email survey will be time consuming as it requires subjects to respond to acquire information
and also detailed information could not be obtained through mail survey.
Interview involves direct contact with the subject to acquire in-depth or detailed
information. Interview will involve recording answers from subjects for pre-defined
questionnaires or it will be discussion about list of topics. The person to be interviewed will be
the expert in the field of research or person affected in the field of research. According to
Bryman (2004) there are three types of interview such as un-structured or informal interview and
semi-structured interview which helps in analysis of qualitative data and structured interview
which helps in analysis of quantitative data. Informal interview are flexible and involves open
questions by the interviewer to acquire in-depth information from interviewee. Semi-structured
interview also helps to collect in-depth information from interviewee where interviewer has set
of topics or list of framed questions to be asked to interviewee known as research guide but the
interviewer do not follow the same order of questions as in research guide to every interviewee
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
35
and also interviewer can ask questions related to list of topics which are not listed in research
guide depending on response from interviewee. Structured interview involves interviewer to ask
same set of standard and predefined questions to ask to every interviewee. Focus groups help to
reveal the response of participants for particular situation. It involves more than one participant
to discuss number of topics or theme and interviewing specific participant for particular
situation. Focus groups are a form of group interview in which the participants are allowed to
interact with each other to share their ideas and experiences with researcher simply acting as
moderator or facilitator to collect data based on interaction between participants by monitoring
and recording the interaction.
3.2.3. Observational Research Methods
Observational research method involves observation or case study. Observation is the
simplest research methods and involves observation of group of people by researcher which
mainly helps to provide qualitative data and more detailed information. Observation may be
participant observation or direct observation (Punch, 1998). Participant observation involves
researcher to become participant to monitor the entire context and interact with the subject.
Direct observation involves researcher to be focused on subject to acquire specific information
from subject by monitoring and observing the interaction. Case Study involves detailed study or
analyzes the problem of subject or case through observation or interview. Case study involves
detailed study of real situation of subject or analyzing the situation of subject during different
times to make comparative study. Stake (1994) classified case study in to three types as intrinsic
case study which helps to understand the particular case or subject better, instrumental case study
which involves detailed study of particular subject or case to get insight in to issues associated
with it and to refine a theory based on study and collective case study which involves detailed
study of different cases or subjects to make general or comparative study.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
36
3.3. Research approaches
There are two broader methods of reasoning (Trochim, 2006) as follows
Deductive approach
Inductive approach
Deductive approach informally called top-down approach works from more general
theories to more specific results. Deductive approach begins with thinking about theory related to
any topic or research interest, then hypotheses related to research theory is formulated and
specific hypotheses are tested, then observations are made to collect data related to hypotheses
and finally testing of hypotheses is done by analyzing the collected data to confirm the research
theory. Thus deductive approach is based on deductive thought that allows the researcher to
formulate hypothesis from theory and hypothesis are tested through data collected through
research methods such as surveys and interviews to confirm or reject the theory if the analysis of
data supports hypothesis.
The waterfall representation of deductive approach (Burney, 2008) is
Theory
Hypothesis
Observation
Confirmation
Inductive approach informally called bottom-up approach works from more specific
observations to more general theories (Trochim, 2006). Inductive approach begins with specific
observation, then patterns are discovered or developed, then hypotheses are framed and finally
general theories are framed based on analysis of hypotheses. Thus inductive approach is based
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
37
on inductive thought that allows the researcher formulate tentative hypothesis based on observed
patterns and define a theory by verifying hypothesis through data collected by observation or
experiments.
The hill-climbing representation of inductive approach (Burney, 2008) is
Theory
Hypothesis
Pattern
Observation
This study involves both deductive and inductive approaches.
The deductive approach of this study starts with general about exploratory search, identify
the various techniques suggested by some previous researchers that will support exploratory
search, investigate various existing search engines, exploratory search systems (ESSs) and
techniques to support ESSs and find the specific suitable technique that will support better
exploratory search. The inductive approach of this study intends to start with evaluating search
engines and ESSs using experiment and investigate problems associated with search engine,
ESSs for exploratory search and observe the behavior of participants to find the role of domain
knowledge for exploratory search. Then propose and develop interface design that will support
better exploratory search. Then evaluate interface design by experiment and investigate the
difference between existing system and new interface design to see if interface system support
better exploratory search. Finally frame a theory on use of proposed interface technique for
better exploratory search.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
38
3.4. Research Strategies
According to Bryman, (2004), research strategies can be as follows
Quantitative research
Qualitative research
Quantitative research involves testing of theory by collection of data and uses deductive
approach to derive relationship between theory and research. It involves techniques such as
structured interviews and questionnaires such as closed or open questionnaires (Bryman and
Bell, 2003). According to Punch (2008) quantitative data are information about world collected
through large-scale survey research that are converted in to numbers to generate statistics
through measurements such as counting or scaling. Thus Quantitative research which is
primarily deductive process and objective in approach in which the data collected through facts
and figures are converted in to numerical data which are statistically or mathematically analyzed
to test pre-specified concepts and to draw conclusion on theory or hypothesis based on numerical
data.
Qualitative research involves generation of theory by observation of respondents and uses
inductive approach to derive relationship between theory and research. It involves techniques
such as participant observation, un-structured or semi-structures interviews and focus groups
(Bryman and Bell, 2003). According to Punch (2008) qualitative data are information about
world in the form of words collected through qualitative empirical materials such as observation,
interviews, documents and audio-visual materials. Thus Qualitative research which is primarily
inductive process and subjective in approach attempts to explore behavior and experience of
participants to get in-depth information and generate theory by analyzing data.
3.5. Methods Used in this Research Study
This study involves both quantitative and qualitative research strategies.
Quantitative research of this study uses survey questionnaires to collect data from
participants regarding their various exploratory search tasks, their experience, usage and
preference of various search engines, exploratory search systems (ESSs) and techniques to
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
39
support their exploratory search tasks. Quantitative research of this study also involves
participants in search task in search engines, ESSs and collects data from them using evaluation
questionnaire. It helps to analyze the data gathered from participants in the form of charts, graphs
or tables and find a suitable technique that will support better exploratory search task.
Qualitative research of this study uses interview to collect data from participants regarding
their experience with search engines and ESSs during exploratory search task, the problems
faced by them during exploratory search task, comments and suggestions about search engines
and ESSs. Also observe the behavior of participants to find the role of domain knowledge for
exploratory search. Based on review of existing literature, by analyzing data collected by survey
questionnaire, experiment evaluation questionnaire and based on opinions from participants
during interview to develop an alternative interface to support better exploratory search. Then
interview participants to collect qualitative data from them about alternative interface after
involving participants in the evaluation of alternative interface. It helps to enhance the proposed
interface based on feedback from participants and propose a theory based on analysis of
information.
3.6. Procedure
A number of participants were engaged in survey about their experience of exploratory
search task, problems faced by them for exploratory task, their experience, usage and preference
of various search engines, exploratory search systems (ESSs) and techniques to support their
exploratory search tasks. The participants also surveyed about their opinion about techniques
suggested by previous researchers that will support exploratory search to find from them which
technique will support better exploratory search task.
The participants also engaged in exploratory searching process in search engine Google
and social tagging system (ESS) Delicious. The domain knowledge of participants related to task
was analyzed by engaging participants in knowledge questionnaire session about domain of
taken task. Then all the participants were engaged in keyword searching process of particular
exploratory task. Some of them lack in domain knowledge of task that was taken to search.
Remaining had good domain knowledge of search task. After completing the search task, the
participants were engaged in evaluation questionnaire session about their feedback on Google
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
40
and Delicious. The participants were also interviewed about their experience with Google and
Delicious during exploratory search task, the problems faced by them during exploratory search
task, comments and suggestions for enhancements in Google and Delicious to support better
exploratory search. The difference in searching behavior of participants with and without domain
knowledge of topic was identified by analyzing the number of URLs they have visited during the
search task, time they have taken to complete the task, way of search behavior, whether they are
able to complete the task. The participants were engaged in evaluation of alternative interface
developed after analyzing the results gathered from previous study methods. The participants
were interviewed for their feedback about alternative interface to find how far the designed new
interface made their exploratory keyword search better compared to the former two search
systems.
3.7. Questionnaire Design
Questionnaire can be used for range of activities or research which helps to form
background of research or activity by collecting data from large group of people to address the
research problem and objective which can be conducted through face-to-face questionnaire or
through self-completion questionnaire such postal or online surveys. Questionnaire helps to
provide, gather and analyze data efficiently and accurately.
3.7.1. Self-completion Questionnaire
Self-completion questionnaire also called as self-administered questionnaire involves many
forms of sending questionnaire such as postal questionnaire or mail questionnaire to large
number of people at relatively low cost (Bryman, 2004). Self-completion questionnaire are cheap
to conduct which can be distributed to wide geographic area in less time and more respondents
can answer the questions themselves by completing the questionnaire in convenience of their
time.
The questionnaire used in this research is self-completion questionnaire through online
surveys as this research involves gathering data from participants regarding their opinions,
awareness and perceptions about exploratory search, exploratory search systems (ESSs) and
techniques that support exploratory search and ESSs. Each participant can complete the
questionnaire in their own time which is easy for them to understand and answer the questions.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
41
3.7.2. Question Styles
Questions can be presented in following two formats relevant to self-completion
questionnaire and structured interview research design (Bryman, 2008)
Open Questions
Closed Questions
Open Question allows the participant to use their own words for questions which helps to
collect qualitative data. Participant will take more time to answer open questions but they are
free to express their own desire and interest so that wide range of data responses can be
collected. Thus knowledge level of participants and new areas can be explored through open
questions.
Closed questions allows the participant to answer the questions from the options provided
which helps to collect quantitative data. Closed questions will be easy for participant to answer
but they are restricted to options provided so that responses may not be true. Closed questions
are hard to design but it is easier to administer data collection through closed questions and the
data collected can be easily processed and analyzed. Trochim (2006) explained different
approaches of closed questions such as:
Dichotomous questions – Questions involving two way responses such as yes/no,
true/false and agree/disagree response.
Questions based on Level of Measurement – Nominal questions which have numbering
response option and Ordinal questions that are based on ranking the response based on
preference or importance of participant.
Questions based on interval level Measurement – Likert response scale that involves
questions based on 5-points scaling response, Semantic differential scale that involves
questions based on scaling with bipolar adjectives response pairs and Guttman scale
which involves participants to checklist each response they agree.
Filter or Contingency Questions – Questions that involves filtering subsequent questions
and avoid subsequent questions by participants based on response to previous question.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
42
3.7.3. Presentation of Questionnaire
Better organization of questionnaire will help to gain information needed from
participants. Questions should be presented in professional and attractive format to make easy for
the participants to fill the answer. Questionnaires should be framed relevant to research objective
with simple language which will be easy for participants to read and understand. The font of
questions should be large enough to read by participants, instructions accompanying the
questions should be clear, questions and pages should be numbered and if possible should be
grouped in to sections and should be accompanied by pictures to support questions. The survey
questionnaire should have introduction about the research objective, should start with questions
that are important to research objective, start with questions that will attract interest of
respondents and should end up with greeting to participants. Byman (2008) pointed out rules for
designing questions such as researcher while framing the questionnaire should bear in mind the
research questions, what information they need to know or obtain and how the participant will
answer the question. Also the researcher while framing questionnaire should avoid ambiguous
terms, avoid long questions and be specific, avoid jargons, double-barreled questions and
colloquialisms, avoid very general questions, avoid leading questions, avoid questions that
include negatives, minimize bias, avoid technical terms and ensure options are mutually
exclusive.
The self-completion survey questionnaire used in this research study are revised several
times and presented in well structured format that will be easy for respondent to understand and
answer. Questions are ordered using numbers and clearly spelled in simple language to make
easy for participant to answer. Closed questions such as multiple choice questions are used in
which participants need to select a single answer from number of choices and some questions
require participant to answer straight forward among two choices provided such as yes or no
questions.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
43
3.8. Pilot Study
Pilot Study is preliminary investigation that helps to test the research methods whether the
research methods fit for the research to be carried on, whether the research is possible and can be
conducted without any difficulties. Pilot Study helps to find the needs of research, any changes
or amendments to be done to research methods to ensure smoothness during research. In case of
questionnaire method, pilot study helps to test the design of questionnaire whether the
questionnaire and instructions can be clearly understand by respondents, how long it will take for
them to respond the questionnaire and whether any difficulties faced by them while answering
the questionnaire. Also it will help to assess the questionnaire design and it will help to
understand the difficulties in any question and whether the response to questions will be as
expected. Thus pilot study guides to frame efficient questionnaire that will help to collect quality
data. In case of evaluation using experimental design, pilot study helps to ensure the
effectiveness of design and to ensure that the design working as expected. Pilot Study should be
conducted similar to face-to-face interviews to observe the reaction of participants and comments
of the respondent for each question.
Pilot study of this research study was conducted after designing survey questionnaire,
experiment evaluation questionnaire and alternative interface design. Questionnaire and interface
design was tested to ensure smoothness in research by finding efficiency of questionnaire, design
and find whether all work as expected and planned. Two of my friends attended the pilot study
and pointed out problems faced by them in understanding and responding both the survey and
evaluation questionnaire and in using interface design. According to pilot study and based on
valuable suggestion and comments of my supervisor, modifications were done to questionnaires
and interface design.
3.9. Sampling
All research study involves selection of samples from population to collect data to
investigate research topic. Blaikie (2004) explained the use of sampling for research as the
sampling reduces the cost and time of research study as studying whole population will be
tedious and expensive and ideal sample should reflect the whole population to help analyze and
make judgment about whole population. According to Bryman (2004) there are two main types
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
44
of sampling such as probability sampling and non-probability sampling. Probability sampling
gives every unit or element in a population a known chance of being selected but non-probability
sampling does not provide chance of being selected. Also Bryman (2004) classified probability
sampling in to four types such as simple random sampling, systematic sampling, stratified
random sampling and multi-stage cluster sampling and classified non-probability sampling in to
three types such as convenience sampling, snowball sampling and quota sampling.
This research study has employed snowball sampling. As this research study involves
selection of participants relevant to research topic and based on research questions, snowball
sampling approach was used which involves selection of people relevant to research and making
contact with those people to collect data (Bryman, 2004).
3.10. Participants Recruitment
This research study aims to provide better interface for users of internet to support their
various search activity. So the participant of this study should include both common less
experienced users and also experienced users of internet so that data can be collected from them
regarding problem faced by them during their search activity and can be analyzed to provide
solution in the form of better interface common to all types of users. A total of sixteen
participants were recruited for this study with different age, gender including postgraduate
students with major in information science related subjects, students with major in other subjects,
professionals working related to information field and other professionals working in other
fields. The sixteen participants are invited to complete online survey questionnaire, attended
experiment search task and took part in interview.
3.11. Distribution of Survey Questionnaire
Prepared survey questionnaire was distributed in-person, online through email and social
networking site Facebook (www.facebook.com) to selected sixteen participants. Brief
description about research to respondents and they are provided with information sheet.
Information sheet contains details about research and regarding ethics policy of research which
was classified as ‘low risk’ according to university ethics policy. Consent sheet was provided to
every participant for their approval in the form of signature to participate in the research by
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
45
reading conditions and agreement of participation. Data collected does not contain any
confidential information about respondents and details were presented in information sheet about
how the responses will be processed and published. Responses were obtained from sixteen
participants which was sufficient to analyze data for quantitative study of this research.
3.12. Survey Questionnaire Questions
After careful review of existing literature related to research study, possible questions were
framed and analyzed. After revising the survey questionnaire according to research objective,
existing literature, based on pilot study conducted using my friends and based on comments of
my supervisor, the final questionnaire was prepared and distributed. The structure of survey
questionnaire is as follows:
Question 1: This question asks the participants about how often they use the internet. It helps to
know the usage of the internet by users.
Question 2: This question asks the participants about their experience of exploratory search. It
helps to know whether users experienced any exploratory search.
Question 3: This question asks the participants about the problems faced by them for their
exploratory search. It helps to know various problems faced by users for exploratory search.
Question 4: This question asks the participants whether they are aware exploratory search
systems (ESSs). It helps to know user’s awareness of various ESSs.
Question 5: This question asks the participants what they use for exploratory search. It helps to
know preference of users for exploratory search whether ESSs or search engines.
Question 6 and 7: These questions ask the participants about their ability to use search engine
and ESSs. It helps to know how far they can use search engine and ESSs.
Question 8: This question asks the participants which search engine they prefer for exploratory
search if they have option of choosing only search engines. It helps to know which search engine
users prefer most for exploratory search.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
46
Question 9: This question asks the participants preference of approach for their exploratory
search. It helps to know what type of approach users prefer for exploratory search and will
support their exploratory search
Question 10: This question asks the participants how far existing search engine or exploratory
search systems (ESSs) support their exploratory search. It helps to know how far user’s
exploratory search is supported by search engines and ESSs.
Question 11: This question asks the participants whether they are aware various techniques that
will support exploratory search. It helps to know user’s awareness of various techniques that will
support ESSs and exploratory search.
Question 12: This question asks the participants who aware of various techniques that will
support exploratory search which techniques they feel will support better exploratory search. It
helps to know the technique that support better exploratory search.
3.13. The Experiment
An experiment was conducted to evaluate how far the existing search engines and
exploratory search systems (ESSs) support user’s exploratory search. Also the experiment aimed
to evaluate the role of domain expertise for exploratory search. The Google (www.google.com)
search engine shown in Figure 3.2 and Delicious (www.delicious.com) social tagging system
shown in Figure 3.3 which is ESSs was selected for the experiment. The Google and Delicious
are chosen based on analysis of answers by survey from respondents in which respondents
answered that most of them use Google if they have option of using only search engine for
exploratory search and respondents who are aware of ESSs and use them for their exploratory
search mostly prefer social tagging system such as Delicious for their exploratory search. Also
Google is mostly used search engine and almost every internet users are familiar with Google’s
interface, search options and additional features provided by Google and Delicious is mostly
used social tagging system that provides tag-based search along with keyword-based search.
Thus the experiment was conducted to evaluate search engine Google and Social tagging system
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
47
Delicious for exploratory search task and to evaluate the role of domain knowledge for
exploratory search to find how the people with different level of domain knowledge perform
exploratory search and how the domain knowledge affect their exploratory search. Same sixteen
participants responded for survey questionnaire was used for experiment search task which was
followed by short interview session.
Figure 3.2 – Google
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
48
Figure 3.3 – Delicious
3.13.1. The Exploratory Search Task
The topic taken for exploratory search task was ‘software engineering life-cycle’. This
topic was chosen based on knowledge of participants about topic as this research aimed to
evaluate role of domain expertise for exploratory search. Eleven participants reported to have
good domain knowledge about software engineering while remaining five participants lack
knowledge about software engineering or related fields. The knowledge of participants was
tested using knowledge questionnaire about search topic. The knowledge questionnaire consists
of five questions about software engineering, software engineering life-cycle and related
concepts. All the participants were asked to complete the questionnaire to test their knowledge.
Then all the participants were asked to perform search about ‘software engineering life-cycle’
using Google and Delicious.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
49
3.13.2. Evaluation Questionnaire
After the completion of search task by participants, they were asked to complete usability
evaluation and satisfaction questionnaire that helps to analyze the usability of systems and
satisfaction of participants with the systems. Closed questions based on five point Likert scale
measure was used. Closed questions helps to evaluate participant’s satisfaction and usability of
system based on quantitative data.
3.13.3. Evaluation Questionnaire Questions
The evaluation questionnaire was framed to evaluate Google and Delicious by collecting
participants’ response for following aspects based on their exploratory search task.
Question 1: This question asks the participants whether it was easy to use the system. It helps to
evaluate the ease of use of system.
Question 2: This question asks the participants whether it was easy to learn the use of system. It
helps to evaluate the ease of learn of system.
Question 3: This question asks the participants how effectively they can complete task using
system. It helps to evaluate the effectiveness of system.
Question 4: This question asks the participants about the efficiency of system during search. It
helps to evaluate the efficiency of system.
Question 5: This question asks the participants whether they are satisfied with search results
provided by system. It helps to evaluate the satisfaction with search results provided by the
system.
Question 6: This question asks the participants about the organization of results. It helps to
evaluate the presentation and organization of results by the system.
Question 7: This question asks the participants whether they are satisfied with information
acquired. It helps to evaluate the quality of information provided by the system.
Question 8: This question asks the participants whether they are satisfied with overall system. It
helps to evaluate the overall performance of the system.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
50
3.13.4. Interview
Interview helps to collect qualitative data from participants using open questions.
Interview session consists of questions to find from users the problems faced by participants
while using the systems during experiment, the experience of participants while using the
systems and qualitative comments of the participants regarding systems merits and demerits,
improvements needed in the current systems and enhancements needed in those systems to
support better exploratory search.
3.13.5. Interview Questions
Question 1: This question asks the participants about the problems and difficulties faced by them
when using the search systems for exploratory search task during experiment. It helps to know
the problems and difficulties faced by users while using search systems for their exploratory
search tasks.
Question 2: This question asks the participants about the merits and limitations of the search
systems. It helps to know the various advantages and limitations of search systems.
Question 3: This question asks the participants about the improvements needed for the current
search systems. It helps to identify the necessary improvements needed for current search
systems to support and satisfy search needs of different types of users.
Question 4: This question asks the participants about the enhancements and features needed for
the future development of search systems. It helps to know the enhancements and features
needed in the future development of current search systems to support better and easier
exploratory search of users.
3.14. Alternative Interface for Exploratory Search
After the analysis of results from survey questionnaire and experiment some results can be
drawn. Also by reviewing and analyzing the existing literatures regarding exploratory search and
ESSs some results can be drawn. All of these can act as a base for alternative interface design to
support better exploratory search.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
51
In this research, JavaScript scripting language was used to design HTML based alternative
interface. This research aims to propose new interface by identifying drawbacks in existing
search engines and ESSs to support better exploratory search. This new interface does not
employ any real-time retrieval algorithm or database and ways to enhance those were not
covered in this research study.
3.14.1. Evaluation of Alternative Interface
Same sixteen participants who evaluated Google and Delicious in experiment were used to
evaluate new interface design. It was followed by short interview in which participants were
asked to comment on new interface design about usability of system and satisfaction of
participants with the system and how they feel new interface will support better exploratory
search compared to existing search engines and ESSs such as Google and Delicious evaluated
during previous experiment.
3.15. Data analysis
This research study collects data were collected through three methods such as survey
questionnaire, experiment evaluation questionnaire and interview. The data collected through
survey questionnaire and evaluation questionnaire were analyzed using MS Excel which helps to
perform various statistical calculations and provide diagrammatic representation of results
through various graphs and charts. The data collected through interview and observation were
reviewed manually and discussed in findings.
3.16. Ethical Aspects
This research study was carried out under University of Sheffield ethics policy which was
ethically reviewed under the University of Sheffield’s ethics policy governing research involving
human participants, personal data and human tissue: version 2 (The University of Sheffield,
2011). This research study is classified as ‘low risk’ by research ethics committee. Information
sheet and consent form was provided to the participants. The information sheet contains purpose
of the research along with other details such as method of data collection, processing data and
publication of results. The participants were also informed that participating in research is based
on their own interest and they can withdraw from participation any time they wish. Personal
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
52
details of the participants were not collected and participants were assured that any data collected
from them will be kept confidential, will not be disclosed to any other person under any
circumstances without their consent and it will be used only for data analysis of this research
study. After informing about the details of research study and ethics policy, the participants were
asked to sign the consent form by accepting terms and conditions.
3.17. Practicalities
The main problem associated with this research study is the selection of participants
based on domain knowledge. Also every participant had experience of exploratory search, most
of the participants were not aware of the term ‘exploratory search’. So it took more time for them
to understand the purpose of this research study by reading the provided information sheet. The
practical problems faced during quantitative survey research were framing questionnaires,
sending questionnaires to participants and receiving response from them. Also involving
participants to evaluate search systems and collecting answers from them for evaluation
questionnaire and interview questions following experimental evaluation was found difficult and
time consuming because there was need to meet participants in their free time and explain them
about the usage of social tagging system (ESS) Delicious as most of the users were not aware of
ESSs and its usage. Designing alternative interface by analyzing the results and existing
literature, explaining the participants about alternative interface, involving participants to
evaluate alternative interface and collecting feedback from them were time consuming. The other
practical problems associated with dissertation format of dissertation and duration for completing
dissertation.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
53
CHAPTER 4
RESULTS AND
FINDINGS
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
54
4.1. Survey Questionnaires
The survey questionnaire focused on following aspects of research:
Participants’ usage of internet and experience of exploratory search
Participants’ experience with search engines, exploratory search systems
and usage of them for exploratory search
Participants’ preference of various approaches and techniques to support
exploratory search
Usage of Internet
1. How often do you use the internet?
Obligations Response Count Response Percentage
Daily 16 100%
Several times a week 0 0%
Weekly 0 0%
Several times a month 0 0%
Monthly 0 0%
Never 0 0%
Table 4.1
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
55
0
20
40
60
80
100
120
Daily Several times a
week
Weekly Several times a
month
Monthly Never
Usage of Internet
Daily
Several times a week
Weekly
Several times a month
Monthly
Never
Figure 4.1
This question helps to know the usage of the internet by people. From the above graph
shown in Figure 4.1 we could find that every participant uses the internet daily. This states that
almost all people use the internet daily for their various day-to-day activities and information
needs.
Experience of Exploratory Search
2. Do you have any experience of exploratory search?
Obligations Response Count Response Percentage
Yes 16 100%
No 0 0%
Preferred not to say 0 0%
Table 4.2
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
56
Figure 4.2
This question helps to know the user’s experience of exploratory search. From the above
graph shown in Figure 4.2 we could find that every participant has an experience of exploratory
search. This states that every user have some experience of undefined search goal or unaware of
their search goal which leads to exploratory search. As discussed in literature review most of the
search activity users will have undefined goal, users are unaware and unsure about how and
where to find the information which leads to exploratory search.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
57
Problems faced by users for their Exploratory Search
3. What is the main problem faced by you for exploratory search?
Obligations Response Count Response Percentage
Unclear search goal 7 43.75%
Unaware of how and where to
find the information
5 31.25%
Search system does not
support better exploratory
search
4 25%
Table 4.3
Problems faced by users for their Exploratory Search
Unclear search goal
Unaware of how and where
to find the information
Search system does not
support better exploratory
search
Figure 4.3
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
58
This question helps to know the various problems faced by users for their exploratory
search. From the above graph shown in Figure 4.3 we could find that most of the participants
which count to 43.75% of the participants were found to have unclear search goal. Next to that
31.25% of participants were unaware of how and where to find the information needed to them.
It means users are unaware of which system to use for their search, which path to follow and
how to proceed with their search to attain the information. 25% of participants found the system
they used for their exploratory search does not support better search. This is due to inefficiency
of system to support their search or users find to difficult to use the system and unsatisfied with
the acquired information.
As described in literature review in which Marchionini (2006) explained exploratory
search as which involves both learn and investigate activity and users requires to perform
multiple search session of querying and browsing to acquire required information which cannot
be satisfied by focalized search as explained by JcsCholtes (2010) and search engines along with
advanced search option does not support better exploratory search.
User’s awareness of Exploratory Search System (ESSs)
4. Are you aware of various exploratory search systems?
Obligations Response Count Response Percentage
Yes 7 43.75%
No 9 56.25%
Preferred not to say 0 0%
Table 4.4
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
59
Figure 4.4
This question helps to know the users awareness of various existing exploratory search
systems. From the above graph shown in Figure 4.4 we could find that maximum numbers of
participants are unaware of various exploratory search systems (ESSs) that will support
exploratory search. 43.75% of participants responded that they are aware of some ESSs where
maximum number of participants which counts to 56.25% of participants responded that they are
unaware of any ESSs.
As discussed in literature review, there are many ESSs are available such as information
visualization systems, document clustering systems and intelligent content summarization
systems as described by White et al. (2008) that will support exploratory search but most of the
users are unaware of these ESSs. Some users are aware of some ESSs such as Social tagging
systems and faceted interface systems.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
60
Preference of systems by users for Exploratory Search
5. What do you use for exploratory search?
Obligations Response Count Response Percentage
Search Engines 9 56.25%
Exploratory Search Systems 3 18.75%
Both 4 25%
Table 4.5
Figure 4.5
This question helps to know the preference of users between traditional search engines and
exploratory search systems for their exploratory search. From the above graph shown in Figure
6.5 we could find that most of the participants’ counts to 56.25% prefer only Search Engines for
their exploratory search which can be inferred from previous question as participants who are
unaware of ESSs prefer only Search Engines for their exploratory search. But some participants
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
61
who are aware of ESSs prefer both search engines and ESSs for their exploratory search as 25%
of participants prefer both while 18.75% of respondents prefer only ESSs for their exploratory
search.
As discussed in literature review, exploratory search requires multiple search sessions,
requires users to refine and use multiple queries during iterative search and requires them to
obtain cues about their next step to navigate search results. Thus some users who are aware of
social tagging system such as Delicious use them for their exploratory search in which tags from
other users acts as navigation cues that facilitate exploratory search of users as explained by
Millen et al. (2007). Some users felt that faceted interface system with category search option as
used by them in most of the systems such as library system support their exploratory search
which proved the study of Shneiderman and Kules (2008) to determine better information access
and retrieval provided by faceted search interface. Users who are unaware of ESSs preferred
only traditional Search engines for their exploratory search but most of them face the problems
as discussed during second question as keyword search and query formulation provided by
search engines are complex and difficult as discussed in literature review.
Ability to use Search Engines
6. If you use search engine for exploratory search, how would you describe your ability to
use them?
Obligations Response Count Response Percentage
Beginner 0 0%
Intermediate 0 0%
Experienced 3 23.08%
Expert 10 76.92%
Table 4.6
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
62
0
10
20
30
40
50
60
70
80
90
Beginner Intermediate Experienced Expert
Ability to use Search Engines
Beginner
Intermediate
Experienced
Expert
Figure 4.6
This question helps to know the ability of participants to use search engine. From the
above graph shown in Figure 4.6 we could find that most of the participants as a percentage of
76.92% of participants are expert in using search engines. Some participants counts to 23.08% of
participants preferred they are only experienced as they are not aware using some options
provided by search engines. As discussed in literature review, even the expert users of search
engines told that search engines does not support their search needs.
Ability to use Exploratory Search Systems (ESSs)
7. If you use exploratory search systems for exploratory search, how would you describe
your ability to use them?
Obligations Response Count Response Percentage
Beginner 0 0%
Intermediate 4 57.14%
Experienced 3 42.86%
Expert 0 0%
Table 4.7
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
63
0
10
20
30
40
50
60
Beginner Intermediate Experienced Expert
Ability to use Exploratory Search Systems (ESSs)
Beginner
Intermediate
Experienced
Expert
Figure 4.7
This question helps to know the ability of participants to use exploratory search systems
(ESSs). From the above graph shown in Figure 6.7 we could find that 42.86% of participants
preferred that they are experienced in using ESSs. Remaining 57.14% of participants preferred
that they are only intermediate in using ESS they use. As discussed in literature review, even the
experienced users of ESSs told that ESS does not support better exploratory search and most of
the ESSs are tough to handle for their search needs.
Preference of Search Engines by users for Exploratory Search
8. If you use Search engine, which one do you use most for exploratory search?
Obligations Response Count Response Percentage
Google 10 62.5%
Bing 1 6.25%
Yahoo 3 18.75%
Live Search 1 6.25%
Others 1 6.25%
Table 4.8
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
64
Figure 4.8
This question was asked to find the users preference of search engine if they have option
of using only Search Engines for their exploratory search. From the above graph shown in Figure
4.8 we could find that most participants as count of 62.5% of participants prefer Google for their
exploratory search. They felt that search option along with advanced search option and search
results provided by Google was better than any other search engines. Next to Google, 18.75% of
participants preferred Yahoo which is also most used along with Google. The users those
preferred Google and Yahoo felt that they were most used because of the additional features
provided by them such Mail, News, Videos and images provided by them. Each of Bing and
Live Search which was formerly known as MSN search was used by one respondent which is
6.25% of total participants. One participant preferred other search engine option as the
participant was found using search engines such About.com, Ask and Answers.com.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
65
Approach to support user’s exploratory search
9. Among the following which type of approach do you prefer most for exploratory search?
Obligations Response Count Response Percentage
Content-based search 3 18.75%
View-based search 5 31.25%
Visualization of search results 2 12.5%
Query suggestions 6 37.5%
Table 4.9
Approach to support user's exploratory search
Content-based search
View-based search
Visualization of search results
Query suggestions
Figure 4.9
This question helps to know the users preference of approach provided by various systems
that support and made their exploratory search better and easy. From the above graph shown in
Figure 4.9 we could find that 18.75% of participants’ preferred content-based approach, 31.25%
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
66
of participants preferred view-based search, 12.5% of participants preferred visualization of
search results and most of the participants’ counts to 37.5% of participants preferred query
suggestions provided by some systems make their exploratory search easy.
The participants who preferred content-based search approach for exploratory search felt
that options to construct and refine their query based on content-based approach will make their
exploratory search easier. As discussed in literature review, content-based approach is supported
by systems that provide query by example search method such as visualization system developed
by Aurnhammer et al. (2006) called TagSphere that helps in content-based image retrieval by
selecting example images from suggestion display and clustering system developed by Wang et
al. (2007) called IGroup that helps users to refine their search results by choosing suitable image
from cluster corresponding to their search query. The participants who preferred view-based
search felt that view-based search will help in easier exploratory search because of the graphical
interaction provided by them. Also some participants felt that view-based search provided by
faceted-interface with multiple category search option that make more easier even for users
without proper domain knowledge. The participants who preferred visualization of search results
approach explained that visualization of search results will help them easy navigation of search
results in information space to explore through their desired path which is used in most of digital
library systems such as the one developed by Seifert and Kruppa (2010). The participants who
preferred query suggestion approach for exploratory search were found to be users of social
tagging system as they provide query suggestions in the form of tags suggested by other users the
provide information cues to users. This proved the study of Krause et al. (2008) as discussed in
literature review that social tagging systems provide better information seeking environment and
effective exploratory search.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
67
Support by Search Engines or Exploratory Search Systems for Exploratory Search
10. How far existing Search engines or exploratory search systems support exploratory
search?
Obligations Response Count Response Percentage
1 – Bad 3 18.75%
2 – Fair 5 31.25%
3 – Good 6 37.5%
4 – Very Good 2 12.5%
5 – Excellent 0 0%
Table 4.10
Figure 4.10
0
5
10
15
20
25
30
35
40
1-Bad 2-Fair 3-Good 4-Very Good 5-Excellent
Support by Search Engines or Exploratory Search System
for Exploratory Search
1-Bad
2-Fair
3-Good
4-Very Good
5-Excellent
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
68
This question helps to know how far the existing Search engines or Exploratory Search
Systems support user’s exploratory search. From the above graph shown in Figure 4.10 we could
find that 18.75% of participants preferred option 1 which corresponds to bad support, 31.25% of
participants preferred option 2 which correspond to fair support, 37.5% of participants preferred
option 3 which correspond to good support, 12.5% of participants preferred option 4 which
correspond to very good support and 0% of participants preferred option 5 which correspond to
excellent support provided by Search Engines or ESSs for exploratory search.
The participants who preferred options 1 and 2 were found to be mostly the users of
Search engines which provide less support compared to ESSs as they provide information to
satisfy only less number of searchers. The participants who preferred options 3 and 4 were found
to be mostly the users of ESSs. The two participants who preferred option 4 of very good support
were found to user’s social tagging system for their exploratory search.
User’s awareness of various techniques to support exploratory search
11. Are you aware of various techniques that will support exploratory search?
Obligations Response Count Response Percentage
Yes 8 50%
No 8 50%
Preferred not to say 0 0%
Table 4.11
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
69
Figure 4.11
This question helps to know the users awareness of various techniques that will support
exploratory search. From the above graph shown in Figure 4.11 we could find that half of
participants are aware while remaining half of participants are unaware of techniques that will
support exploratory search.
As discussed in literature review there are many techniques provided by various
exploratory search systems and search engines that will support exploratory search but only half
of the participants are aware of those techniques.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
70
Technique to support better exploratory search
12. If answered yes to above question, which technique you think will support better
exploratory search?
Obligations Response Count Response Percentage
Faceted Interface 2 25%
Visualization 1 12.5%
Clustering 1 12.5%
Semantic web 1 12.5%
Tags 3 37.5%
Table 4.12
Figure 4.12
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
71
This question helps to know the preference of users to support exploratory search among
respondents who are aware of various techniques to support exploratory search. From the graph
shown in Figure 4.12 we could find that 25% of participants preferred interface technique that
will support exploratory search. Each of visualization technique, clustering technique and
semantic web technology is preferred by one participant. Most of the respondents which count to
37.5% of participants among the participants who are aware of techniques to support exploratory
search preferred tags to support exploratory search. So it is clear that most users are using
systems that contains tags and faceted interface to support exploratory search which the
participants felt that systems with tags and faceted interface are easy to use and make their
exploratory search better and easier. As discussed in literature reviews both tags in the form of
information cues from other users and faceted interface with multiple category search options to
select information based on preferences and constraints helps users in easy query reformulation
for effective information seeking in information space that both support users even without
proper domain knowledge to perform better and easier exploratory search.
4.2. Evaluation Questionnaires
The experimental evaluation of Google and Delicious for exploratory search task focused
on following aspect of research
Participants’ satisfaction with existing search engines, exploratory search
systems, their usage and support for exploratory search
Ease of use of system
1 It was easy to use
the system
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
The average ease of use of Google rating was 4.75 with 12 participants rated 5 and
remaining 4 participants rated 4. The average ease of use of Delicious rating was 4.38 with 9
participants rated 5, 4 participants rated 4 and remaining 3 participants rated 3. The participants
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
72
found Google was easy to use compared to Delicious as Google was mostly used search engine
but some participants felt that advanced search options and some features provided by Google
are difficult to use. Most of the participants found difficult to use Delicious as they claim that
they are first time users of Delicious.
Ease of learn of system
2 It was easy to learn
the use of system
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
The average ease of learning of Google rating was 4.63 with 10 participants rated 5 and
remaining 6 participants rated 4. The average ease of learning of Delicious rating was 4.25 with
8 participants rated 5, 4 participants rated 4 and remaining 4 participants rated 3. The participants
felt that Google was easy to learn compared to Delicious similar to ease of use of Google. Some
participants felt that some features provided by Google needs more practice to learn and most
participants felt that who were first time users of Delicious felt it was difficult to learn most of
the functions.
Effectiveness of system
3 I can effectively
complete task
using system
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
The average effectiveness of Google score was 3.88 with 6 participants rated 5, 4
participants rated 4, 4 participants rated 3 and remaining 2 participants rated 2. The average
effectiveness of Delicious score was 4.13 with 7 participants rated 5, 5 participants rated 4, 3
participants rated 3 and remaining 1 participant rated 2. Participants felt that completion of task
was better and can be done effectively using Delicious compared to Google because they felt that
tags acting as navigation cues are more useful than search results which are links provided by
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
73
Google. As discussed in literature review tags associated with every object helps to access and
retrieve them effectively.
Efficiency of system
4 Efficiency of
system is good
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
The average efficiency of Google score was 4.06 with 6 participants rated 5, 6 participants
rated 4, 3 participants rated 3 and remaining 1 participants rated 2. The average efficiency of
Delicious score was 3.94 with 6 participants rated 5, 5 participants rated 4, 3 participants rated 3
and remaining 2 participants rated 2. Participants felt that efficiency of system was same with
both the systems with some participants felt Google was slightly efficient than Delicious in some
cases.
Satisfaction with search results
5 Satisfied with
search results
provided by
system
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
The average satisfaction with search results provided by Google rating was 3.81 with 5
participants rated 5, 6 participants rated 4, 2 participants rated 3 and remaining 3 participants
rated 2. The average satisfaction with search results provided by Delicious rating was 4.0 with 6
participants rated 5, 5 participants rated 4, 4 participants rated 3 and remaining 1 participant
rated 2. Participants felt that search results provided by Delicious are better than Google because
Delicious provides tags from other users that help to refine their search query while Google
provides only links to sites. As discussed in literature review, tags from other users acts as
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
74
navigation cues that helps the users to browse through bookmarks according to time and popular
bookmarks.
Presentation and organization of results
6 Organization of
results is good and
useful
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
The average presentation and organization of results by Google rating was 3.69 with 4
participants rated 5, 6 participants rated 4, 3 participants rated 3 and remaining 3 participants
rated 2. The average presentation and organization of results by Delicious rating was 3.62 with 4
participants rated 5, 5 participants rated 4, 4 participants rated 3 and remaining 3 participants
rated 2. Participants found that presentation and organization of results are better in Google
compared to Delicious because Google provides results ranked in some order and some
participants felt difficult to access the results provided by Delicious.
Quality of information
7 Satisfied with
information
acquired during
search
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
The average quality of information provided by Google score was 3.19 with 3 participants
rated 5, 3 participants rated 4, 5 participants rated 3, 4 participants rated 2 and remaining 1
participants rated 1. The average quality of information provided by Delicious score was 3.5 with
4 participants rated 5, 4 participants rated 4, 5 participants rated 3, 2 participants rated 2 and
remaining 1 participants rated 2. Most participants are satisfied with information acquired using
Delicious than Google as they are able to find exact information using Delicious but they felt that
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
75
they need to analyze the results and information provided by Google to acquire information
needed to them. As discussed in literature review, tags interpreted as folksonomies acts as
navigation cues that help to learn and acquire better knowledge that facilitate better information
acquisition to find exact and correct information.
Overall performance of system
8 I am overall
satisfied with
system
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
The average overall performance of Google score was 3.93 with 5 participants rated 5, 6
participants rated 4, 4 participants rated 3 and remaining 1 participants rated 2. The average
overall performance of Delicious score was 3.81 with 4 participants rated 5, 7 participants rated
4, 3 participants rated 3 and remaining 2 participants rated 2. In case of overall performance,
participants felt that Google was better compared to Delicious because they felt that Google was
used often that satisfy most of their information needs and more features are provide by Google.
4.3. Interview
Interview helped to acquire qualitative information from participants regarding the
problems faced by participants during exploratory search task, experience with Google and
Delicious during exploratory search, advantages and disadvantages of Google and Delicious,
improvements needed in those systems and enhancements and features needed in the future
development of those systems to support their exploratory search better. The interview was
structured using following four questions to collect qualitative information from participants.
1. What are the problems and difficulties faced by you when you are using the search systems for
exploratory search task?
2. What are the merits and limitations of the search systems?
3. What are the improvements needed for the current search systems?
4. What are the enhancements and features needed for the future development of search systems?
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
76
Participants are satisfied with simplicity and ease of Google but 12 out of 16 participants
told that the met with difficult of forming search queries and to analyze the results to navigate
their next step. Also they felt that search results provided by Google are just ranked in some
order and they felt it would be better if the search results are classified. 6 out of 16 participants
told that they felt difficult in analyzing and navigating the search results. Among them five
participants told that they felt very difficult in navigating the search results for their next step due
to lack of domain knowledge of search task. 10 out of 16 participants were not able to complete
the task as they felt Google does not provide effective search results and information.
Participants also felt that advanced search option provided by Google was quite easy to use but it
does not provide better search results and support better information acquisition. A participant
told that “Advanced search option provided by Google does not make any difference in
information acquisition”. Participants told that many irrelevant and unwanted search results are
provided by Google so there is need to navigate number of result pages to find information
suitable to them. Two participants told that need of navigation for number of pages for
acquisition of desired information makes lose their interest to search. 75% of participants
suggested improvements for Google that it would support better search if they are provided with
better search options, provide more relevant and accurate results, classification of search results
and query suggestions by semantic interpretation of user search query.
In case of Delicious, 9 out of 16 participants told that met with many problems while using
the system as they beginners in using the system. 13 out of 16 participants told that use of system
and learning of system is difficult. Participants told that tags helps to refine query and support
better browsing but they told tags should be accompanied with better description. A participant
told “query suggestions in the form of tags helped me to complete my search soon”. Participants
also told that a tag with different meaning and use of tags in different languages provides
irrelevant results. Five participants told they are very confused with use of tags as same keyword
with different meaning provides different results. 81.25% of participants suggested some
improvements for Delicious such as it should be provided with more search options, standard
and good description of tags and classification of search results.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
77
4.4. Role of Domain Knowledge for Exploratory Search
The role of domain knowledge for exploratory search was analyzed using the number of
URL visited during their exploratory search task, way of search behavior and based on
completion of task. Eleven participants who are domain experts with good domain knowledge
about search topic were found to be better in terms of search efficiency to find the relevant
information than five participants who are domain novices with least knowledge about search
topic. The role of domain knowledge for exploratory search task was also affected by the system
used to search. In case of Delicious systems, experts were found to use keyword based search
queries visited more URLS but novices were found to use tag based search queries thus they
need to visit only less URLs. 5 out of 11 domain experts found to use only keyword search rather
than tag based search. Thus experts were found to use their own search terms rather than search
tags of others as used by novices. In case of Google where the participants had the option of
performing only keyword based search, novices were found to visit more URLs to collect the
information than experts as novices found difficulty to frame correct search query for keyword
search. The average number of URLs visited by domain novices was found to be 21 whereas the
average number of URLs visited by domain experts was found to be 7. Thus experts were found
to depend more on their domain knowledge but novices found to depend on the knowledge of
others. Experts were found to better in terms of efficiently completing the task and in better
acquisition. 81.82% of domain experts are found to be effective in terms of search task
completion as opposed to 40% of domain novices. Experts were found to acquire more relevant
and accurate information but novices were found to acquire only general information. Thus the
domain knowledge plays a significant role for exploratory search but it also depends on the
system used for search task.
4.5. Summary of the Findings
The data collected through quantitative methods such as survey interview, experiment
evaluation questionnaire and qualitative method interview were analyzed and results were
summarized as follows that acts as base for alternative interface design.
People use internet daily for their various information needs and everyone
have some experience of exploratory search because at least some of the
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
78
search activity of people will have unclear search goal and they are unaware
of where to find the information and unsure about how to find the
information.
For exploratory search activity, most of the users face the problem of unclear
search goal while remaining users face the problem of unaware of how and
where to find the information or found their search systems does not support
better exploratory search.
Most of the users were not aware of exploratory search systems (ESSs) and
they use only search engines for their exploratory search found to be expert
and experienced users of search engines. Some users are aware of some
ESSs were found to intermediate and experienced users of ESSs. Among the
search engines, Google was used by most of the users for their exploratory
search. But both the users of search engines and ESSs found their search
systems does not support better exploratory search to satisfy their
information needs.
Most of the users prefer approaches such as query suggestions as provided
by social tagging systems and view-based search as provided by faceted
interface systems for their exploratory search.
Most of the users are not aware of techniques to support exploratory search
and among the users who were aware of techniques to support exploratory
search, most of them preferred tags and faceted interface.
The search engine Google and social tagging system (ESS) Delicious got
highest rating from users for their easy to use but Google was found to be
more easier to use than Delicious by users. Also both systems were found
easy to learn by users but Google was found to be much easier to learn than
Delicious.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
79
Delicious helps users to effectively complete their exploratory search task
than Google. Similarly Delicious was found to be better than Google in
terms of providing results for user search query and in providing quality of
information to satisfy users search needs.
Users found the efficiency of system and presentation and organization of
results are at equal level with both Google and Delicious but Google was
found slightly better than Delicious by some factors. In case of overall
performance users are more satisfied with Google than Delicious.
Users felt difficulty in forming query words for their Google search and they
found query suggestions provided by Delicious in terms of tags helps to
refine their search query better and helps in easy navigation of search results.
Google was found to be ineffective by users in terms of exploratory search
task completion and problems faced by users with Google are ranked order
of search results provided for users search query, unwanted results and
results irrelevant to users search query.
Most of the users are found to be beginner or intermediate users of Delicious
so they found difficult to use and learn Delicious and they face the problem
with description, meaning of tags and use of tags in different languages.
Users felt that more search options and classification of search results by
both systems will support their exploratory search better.
The domain knowledge of users play an important role for their exploratory
search but it also depends on search systems they use for their exploratory
search that will reduce the role of domain knowledge for exploratory search
task.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
80
4.6. Alternative Interface for Exploratory search
Based on analysis of statistical results and experiment results and evaluation of existing
exploratory search systems plus the review of existing literature of exploratory search,
exploratory search systems (ESSs) and techniques to support exploratory search and ESSs, an
alternative interface was designed to support better exploratory search. The new interface was
designed using JavaScript scripting language and it does not employ any real time database or
real time algorithm to retrieve information. The interface was designed based on Google and
social tagging systems because the analysis of statistical results from survey and experimental
evaluation results shows that search engines was used by most of the participants for their
exploratory search in particular Google search engine was used by most of the participants which
they found to be easy to use but participants suggested some approaches provided by social
tagging systems will support their exploratory search better. The general layout, search options
and features provided by new interface was discussed below.
4.6.1. General Layout
The main page of the interface shown in Figure 4.13 is similar to that Google search
results page with options for filtering results by preferences such as images, videos, news,
shopping, blogs, books, places, discussions and patents at the left side of the interface, link to
advanced search option near search box and search results below the search box.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
81
Figure 4.13 – Alternative interface: General Layout
4.6.2. Search Options
Three types of search options shown in Figure 4.14 are provided in the interface design
such as normal search option, search option for searching all results or current results provided
near search box and advanced search options.
4.6.3. Keyword Suggestions
The search results page is provided with keywords of user search query along with some
related keyword suggestions that helps users to refine their query to more specific using keyword
suggestions to get more specific desired results.
4.6.4. Search within Results
Search option provided at the bottom of the interface is provided with two options such as
to search all results or to search within current results that help the user to refine their query
using keyword suggestions and submit their own different query to search all results or to search
current results.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
82
Figure 4.14 – Alternative interface: Search Options
4.6.5. Advanced Search Options
Advanced search option shown in Figure 4.15 consists of two parts such as options to
place constraints on search and facet category search. The first part of search helps the users to
place constraints on the search query similar to Google advanced search option using different
keyword or more than one keyword, exact phrase or sentence, file type and language. The second
part consists of facet category search that helps the users to refine their search using different
facets such as person, locations, period and themes.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
83
Figure 4.15 – Alternative interface: Advanced Search Options
4.7. Evaluation and User Feedback on Alternative Interface
The same sixteen participants who participated in survey and experiment evaluation of
search systems were asked to evaluate the alternative interface for exploratory search. After the
evaluation they were interviewed to collect feedback from them on the new interface design.
Every participant found the general layout which was designed similar to Google’s search result
page easy to use and learn as of Google. Participants felt that different types of search options
provided in the interface made their searching simplified and easier. 75% of participants
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
84
commented that keyword suggestions along with search keywords helped them to choose the
desired keywords to refine their search queries that made their browsing easier and faster. But
four participants explained that it would be better if the interface provides more accurate and
related keywords corresponding to their search query keywords. Almost all participants are
satisfied with search within current results option and said that the search option for searching
current results helped in easy navigation of search results and refine the search results by
searching using different keywords.
Every participant is satisfied with the advanced search and found it very useful
particularly the faceted search option. 14 out of 16 participants told that the faceted category
search option was very simple to use which is also more useful to navigate the search by
selecting the desired facet category. 6 out of 16 participants felt that the advanced search option
to place constraints on search query could have more options to refine the search query to get
more desired results. A participant told that “Faceted category was better than advanced search
option of placing constraints on search query that makes searching easier”. Thus the new
interface design was found more useful and better than current existing search systems
particularly Google and Delicious taken for exploratory search task experiment by all 16
participants. Thus the summary of participants feedback showed that all participants regarded
that the search option, keyword suggestion, advanced search and the overall alternative interface
design would support and make their exploratory search better and easier.
4.8. Limitations of the study
The main limitation of this research study was the small sample of population used for
data collection. If more time and resources are available, more participants with different internet
user level, different age and with different profession based on domain knowledge could have
been recruited to involve in this research study. The overcome of this limitation would have let
to the development of better alternative interface to satisfy all types and level of users. Most of
the users were not aware of any exploratory search systems. During the evaluation of Delicious
during experiment, most of the users who were not aware of ESSs were not aware how to use the
Delicious for their exploratory search. Even though they were taught how to use it, they were not
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
85
given more time to learn better due to time limitation. The overcome of this limitation would
have let to more determined evaluation results. Also due to time and resources limitation only
one of the search engine Google and one ESS Delicious was taken for experimental evaluation to
satisfy the evaluation of existing search systems. The overcome of this limitation would have let
to more findings that would have acted as a base for better alternative interface design. The
design of alternative interface was also restricted from adding more additional features due to
time limitation.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
86
CHAPTER 5
CONCLUSION
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
87
With rapid growth of World Wide Web, the number of users using internet for most of
their daily activities and search for information for most has grown due to which there is need for
search system to assist all types of users in effective information retrieval. This research study
started with review of existing literature regarding exploratory search, search engines and
exploratory search systems (ESSs), various techniques and approaches to support exploratory
search. These all acted as a basic for design of survey questionnaire, evaluation questionnaire
following experimental evaluation of search engine and ESS and interview. After analyzing the
results collected through above data collection methods, some important results and findings
were summarized. These along with review of existing literature acted as a base for HTML based
alternative interface designed using JavaScript scripting language. The alternative interface was
designed with aim to make users’ exploratory search better and easier. Finally users were asked
to evaluate the alternative interface and provide feedback on the alternative interface for the
support provided by it for their exploratory search.
5.1. Conclusions
The main objective of this research study was to evaluate the existing search systems for
the support provided by it for users exploratory search task and to develop an alternative
interface that support better users exploratory search task, make exploratory search easier with
better search options and efficient information retrieval. Number of quantitative and qualitative
methods was carried out to attain the objectives. First review of existing literature on exploratory
search and existing search systems was carried out. Also approaches and techniques to support
exploratory search was also reviewed.
The survey was conducted to identify the users exploratory search behavior, their
experience of exploratory search, their experience with existing search systems, their awareness,
preference and ability to use existing search systems for their exploratory search tasks and
preference of techniques and approaches to support their exploratory search task. The survey was
carried from participants recruited based on domain knowledge related to objective of this
research study. The analysis of results from survey questionnaire proved that every user use
internet daily and all have some experience of exploratory search. Most of the users (43.75%)
found to have the problem of undefined search goal for their exploratory search. Most of the
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
88
users (56.25%) are not aware of ESSs and they use only search engine for their exploratory
search among which Google was used by most of the users (62.5%). Users who use ESSs for
their exploratory search are found to be intermediate or experienced users as opposite to search
engine users are found to be expert or experienced users. Both the users of search engine and
ESSs found that neither supports their exploratory search better. Most of the users preferred
approaches such as query suggestions (37.5%) and view-based search (31.25%) for exploratory
search and only half of the users (50%) are aware of various techniques to support exploratory
search among them most of them preferred tags (37.5%) and faceted interface (25%). The
analysis and summary of above results and finding helped to find solution for the first research
question ‘What will be the searching behavior of users for exploratory search task?’
The same participants responded for survey were asked to evaluate the existing search
engine Google and social tagging Delicious which was exploratory search system as identified
by past researchers. Following the evaluation, the participants were asked to answer the
evaluation questionnaire to identify the users’ satisfaction with existing search systems and the
support provided by those systems for users exploratory search task. The analysis of results from
evaluation questionnaire showed that users found Google was easy to use and easy to learn with
average user ratings of 4.75 and 4.73 compared to Delicious and any other systems. Also users
are most satisfied with overall performance of Google with average user rating of 3.93. But in
terms of effectiveness of system in helping users to complete their tasks, satisfaction of search
results provided for search query and in providing quality of information to satisfy users’
information needs with average user ratings of 4.13, 4.0 and 3.5, users found Delicious were
better than Google. Users found Google and Delicious are at same level in terms of efficiency
with average user ratings of 4.06 and 3.94 and in organization and presentation of results with
average user ratings of 3.69 and 3.62. The participants were also interviewed to collect
qualitative information from them based on evaluation to know their needs of system for
exploratory search task. The analysis of answers from interview helped to find the users
experience with Google and Delicious, problem faced by participants using those systems for
their exploratory search task, merits and demerits of those systems and enhancements needed in
those systems to support their exploratory search better. The main advantage found by users with
Google was easy to use and easy to learn and with Delicious users found the advantage of query
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
89
suggestions in terms of tags. The problems faced by users with Google were difficulty in query
formation, inefficient for exploratory search task completion, ranked order of results, unwanted
and results irrelevant to users search query and the problems faced by users with Delicious were
description, meaning of tags and use of tags in different languages. The users suggested for
enhancements in Google such as query suggestions, providing more accurate and relevant results
for user search query and in Delicious they suggested it should provide standard and good
description of tags. Also users suggested that both Google and Delicious should provide better
search options and classification of search results. The analysis and summary of above results
and finding helped to find solution for the second research question ‘To what extent the existing
search tools support exploratory search?’
The searching behavior of participants was analyzed to identify the role of domain
knowledge for exploratory search. The analysis of searching behavior of users for exploratory
search task showed that domain knowledge is important for users for their exploratory search but
the role of domain knowledge may be weakened depending on the system used by users for their
exploratory search. The above analysis helped to find solution for the third research question
‘What is the role of domain knowledge in exploratory search, how did it influence the search and
the information collected?’
Based on the results and findings of survey and evaluation questionnaire, an alternative
interface was designed with aim to satisfy users’ need of system to support their exploratory
search better. Same participants used for previous research methods were used to evaluate the
alternative interface and provide feedback on the new interface for the support it could provide
for their exploratory search. The analysis of answers showed that users found the alternative
interface could support their exploratory search better and make it easier. The users are satisfied
with search options, query suggestions and advanced search options such as placing constraints
on search query and faceted category search option and found the above features are more useful
compared to search systems evaluated during experiment. But users also suggested some
improvements such as suggestion of more accurate and relevant keywords by the system. The
analysis of above results helped to find solution for the fourth research question ‘Would the
proposed interface technology make change and support exploratory search behavior?’
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
90
5.2. Implications of the Study
The implications of this research study are
This research study creates awareness about exploratory searching behavior,
search engines, exploratory search systems and search techniques
This study can act as base for development of better real time search system to
support and satisfy information needs of exploratory search
Every people from professionals to common people and from different
geographical area will be interested about this research
5.3. Suggestions for Future Work
This research study could acts as literature for future research related to this study. The
alternative interface along with some suggestions for improvement could act best literature and
can act as base for design of better system. As participants suggested, the interface should
provide accurate and related keywords corresponding to user search query using semantic
interpretation of users search query. Also as suggested by participants, the advanced search
option for placing constraints on user should be designed with more options to help users t find
definite query to filter results to get more accurate and desired results. It would be better for view
based search of facet search category in advanced search option to generate dynamic facets
during runtime based on users search query. For result exploration, it would be better to have
visualization of search results such as textual visualization, graph visualization and visualization
through semantic clusters. Content based search such as query by example for image search will
also provide better search option to query based on examples provided and find similar
documents. The alternative interface was designed only to enhance existing systems by
identifying enhancements in those systems so it does not employ any real time algorithm and
database. So it will be better to develop a real time information retrieval system using real time
algorithm and database to meet real time users’ information needs.
(Word Count: 20493)
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
91
BIBLIOGRAPHY
Aurnhammer Melanie, Hanappe Peter and Steels Luc (2006) Augmenting Navigation for
Collaborative Tagging with Emergent Semantics, Cruz I., Decker S., Allemang D., Preist C.,
Schwabe D., Mika P., (Ed.), ISWC 2006: Proceedings of the 5th International Semantic Web
Conference, LNCS 4273, pp. 58-71, Athens, GA, USA: Springer-Verlag, Berlin Heidelberg.
Bates, M. J. (1989) The design of browsing and berrypicking techniques for the online search
interface. Online Review, 13 (5), 407–424.
Berners-Lee, T., Chen, Y., Chilton, L., Connolly, D., Dhanaraj, R., and Hollenbach, J. (2006)
Tabulator: Exploring and analyzing linked data on the semantic web, Proceedings of the 3rd
International Semantic Web User Interaction Workshop, SWUI06, 2006
Blaikie Norman (2004) Analyzing Quantitative Data, From Description to Explanation, SAGE
Publications, London, 2004
Bryman Alan and Bell Emma (2003) Business Research Methods, Oxford University Press, 2003
Bryman Alan (2004) Social Research Methods, Second edition, Oxford University Press, 2004
Bryman Alan (2008) Social Research Methods, Third edition, Oxford University Press, 2008
Capra Robert G. and Marchionini Gary (2008) The Relational Browser Tool for Faceted
Exploratory Search, JCDL ’08, June 16-20, 2008, Pittsburgh, Pennsylvania, USA.
Chi Ed.H. and Nairn Rowan (2009) Information Seeking with Social Signals: Anatomy of a
Social Tag-based Exploratory Search Browser, CHI 2009, Boston, USA.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
92
Christel Michael G. (2008) Supporting Video Library Exploratory Search: When Storyboards are
not Enough, Proceedings of the 2008 International Conference on Image and Video Retrieval
CIVR ’08, July 7-9 2008, Niagara Falls, Canada.
Cove J. F. and Walsh B. C. (1988) Online Text Retrieval via Browsing, Information Processing
& Management, 24(1), pp 31-37
Ding Wei, Marchionini Gary and Soergel Dagobert (1999) Multimodal Surrogates for Video
Browsing, DL ’99, Proceedings of the Fourth ACM conference on Digital libraries, New York,
USA.
Dork Marian, Carpendale Sheelagh, Collins Christopher and Williamson Carey (2008) VisGets:
Coordinated Visualizations for Web-based Information Exploration and Discovery, IEEE
Transactions on Visualizations and Computer Graphics, 14(6), 1205-1212.
Duggan, G.B., and Payne, S.J. (2008) Knowledge in the head and on the web: using topic
expertise to aid search. Paper presented at the SIGCHI conference 2008, Florence, Italy.
Flick Uwe, (2002), An Introduction to Qualitative Research, Second edition, SAGE Publications,
London.
Fu, W.T. (2008) The Microstructures of Social tagging: a rational model, Paper presented at the
ACM 2008 conference on Computer Supported Cooperative Work, San Diego, CA, USA.
Hearst Marti A., Yee K.P. and Swearingen K. Li. (2003) Faceted metadata for image search and
browsing, Proceedings of the ACM Conference on Human factors in Computing Systems, pp.
401–408, Fort Lauderdale, FL, USA: ACM Press, 2003.
Hemel Sven Van, Paepen Bert and Engelen Jan (2003) Smart Search in Newspaper Archives
Using Topic Maps, ELPUB 2003, From Information to Knowledge: Proceedings of the 7th
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
93
ICCC/IFIP International Conference on Electronic Publishing, University of Minho, Portugal,
25-28 June, 2003.
Hearst Mart A. (2006) Design Recommendations for Hierarchical Faceted Search Interfaces,
ACM SIGIR Workshop, Seattle, WA, USA, August 2006.
Huynh D.F., Miller R., and Karger D. (2007) Exhibit: Lightweight structured data publishing, in
Proceedings of the World Wide Web Conference, pp. 737–746, Banff, Alberta, Canada: ACM
Press, 2007.
Hwang Inbeorn, Kahng Minsuk and Lee Sang-goo (2011) Exploiting User Feedback to Improve
Quality of Search Results Clustering, ICUIMC ’11, February 21-23, 2011, Seoul, Korea.
Hyvonen Eero, Junnila Mikka, Kettula Suvi, Makela Eetu, Saarela Samppa, Salminen Mirva,
Syreeni Ahti, Valo Arttu and Vilijanen Kim (2004) Publishing Museum Collections on the
Semantic Web- The MuseumFinland Portal, WWW2004, May 17-22, 2004, New York, USA.
Indratmo (2010) Supporting Exploratory Browsing with Visualization of Social Interaction
History, University of Saskatchewan Library, Saskatoon, pp. 19-23.
Internet World Stats (2010) Usage and Population Statistics, June 30, 2010, Miniwatts Marketing
Group, Available at: www.internetworldstats.com/stats.htm (Last Accessed on 13/04/2011).
JcsCholtes (2010) Understand the two different faces of search: exploratory search and focalized
search, eDiscovery and information Management, Cae Assessment, Use Enterprise Information
Management Principles for eDiscovery, Litigation Readiness and Early, Available at:
http://zylab.wordpress.com/2010/03/26/understand-the-two-different-faces-of-search-
exploratory-search-and-focalized-search/ (Last Accessed on 21/06/2011).
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
94
Kammerer, Y., Nairn, R., Pirolli, P., and Chi, E.H. (2009) Signpost from the masses: learning
effects in an exploratory social tag search browser, Paper presented at SIGCHI conference 2009,
Boston, USA.
Kerr Bernard, Millen David R. and Feinberg Jonathan (2006) Dogear: Social Bookmarking in
the Enterprise, CHI 2006 Proceedings, Social Computing 1, April 22-27, 2006, Montreal,
Quebec, Canada.
Krause, B., Hotho, A., and Stumme, G. (2008) A Comparison of Social Bookmarking with
Traditional Search, Advances in Information Retrieval, ECIR 2008, LNCS 4956, pp. 101-113,
2008
Kules Bill, Capra Robert, Banta Matthew and Sierra Tito (2009) What do Exploratory Searchers
Look at in a Faceted Search Interface?, Joint International Conference on Digital Libraries
(JCDL), 2009, Austin, Texas.
Marchionini, G. (1997) Information Seeking in Electronic Environments, Cambridge University
Press, June 1997.
Marchionini, G., Brunk, B., Komlodi, A., Conrad, F., and Bosley, J. (2000) Look before you
click: A relation browser for Federal statistics websites, Proceedings of the ASIS 2000 Annual
meeting, Chicago, IL, November 15-18, 2000.
Marchionini Gary (2006) Exploratory search: From finding to understanding, Communication of
the ACM April 2006/ Vol. 49, No. 4.
Mercun Tanja and Zumer Maja (2009) Visualizing for Exploration and Discovery, Libraries in
Digital Age, LIDA 2009, Zadar, Croatia
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
95
Millen, D. R., Yang, M., Whittaker, S., and Feinberg, J. (2007) Social bookmarking and
exploratory search, Paper presented at the ECSCW 2007, Limerick, Ireland.
Mirizzi Roberto and Noia Tommaso Di (2010) From Exploratory Search to Web Search and
back, PIKM ’10, October 30,2010, Toronto, Ontario, Canada.
Moussa Maher Ben, Pasch Marco, Hiemstra, Der Vet Paul Van and Huibers Theo (2006) The
Potential of User Feedback through the Iterative Refining of Queries in an Image Retrieval
System, AMR ’06 Proceedings of the 4th
International conference on Adaptive Multimedia
Retrieval: user, context and feedback, Geneva, Switzerland, July 27-28, 2006.
Mulholland P., Zdrahal Z. and Collins T. (2008) Investigating the Effects of Exploratory
Semantic Search on the Use of a Museum Archive, IEEE 2008 International Conference on
Distributed Human-Machine Systems, 10-12 March 2008, Athens, Greece.
Park Hogun, Myaeng Sung Hyon, Jang Gwan, Choi Jong-wook, Jo Sooran, Roh Hyung-chul
(2008) SketchBrain: An Interactive Information Seeking Interface for Exploratory Search, HCIR
2008, Redmond, WA.
Punch Keith F. (1998) Introduction to Social Research, Quantitative and Qualitative Approaches,
SAGE Publications, London, 1998.
Rasmussen Edie (2008) Characterizing, Supporting and Evaluating Exploratory Search, NSF IS3
Workshop, Chapel Hill, NC, USA
Rudestam (2007) The Research Process, February 2007, Available at: www.sagepub.com/upm-
data/14647_Chapter1.pdf
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
96
Schraefel M.C., Wilson Max, Russell Alistair and Smith Daniel A. (2006) MSPACE: Improving
Information Access to Multimedia Domains with Multimodal Exploratory Search,
Communications of the ACM, Apr2006/ Vol. 49, No. 4.
Seifert Inessa and Kruppa Michael (2010) A Pool of Topics: Interactive Relational Topic
Visualization for Information Discovery, ECDL'10 Proceedings of the 14th European conference
on Research and advanced technology for digital libraries, Springer-Verlag Berlin, Heidelberg.
Shneiderman, B., and Kules, B. (2008) Users can change their web search tactics: Design
guidelines for categorized overviews, Information Processing and Management Journal, Vol. 44,
no. 2, pp. 463-484.
Spink, A. (2002) A user-centered approach to evaluating human interaction with Web search
engines: an exploratory study, Information Processing and Management, 38, 401-426.
Stake, R. E. (1994) Handbook of qualitative research, pp. 236-247, Thousand Oaks, CA: Sage.
Stevens, S.S. (1946) On the theory of scales of measurement, Science, New Series, Vol. 103,
No.2684, Jun 7,1946, pp. 677-680, American Association for the Advancement of Science.
The University of Sheffield (2011) Research Ethics: General Principles and Statements, The
University of Sheffield’s Ethics Policy Governing Research Involving Human Participants,
Personal Data and Human Tissue: Version 2, 2011, Available at:
http://www.shef.ac.uk/content/1/c6/11/15/24/General%20Principles%20and%20Statements%20
vs2.pdf (Last Accessed on 04/05/1011)
Trochim William, M. K. (2006) Research Methods Knowledge Base, Available at:.
www.socialresearch- methods.net/kb/index.php (Last Accessed on 10/05/2011).
Villa Robert, Gildea Nicholas and Jose Joemon J. (2008) FacetBrowser: A User Interface for
Complex Search Tasks, MM ’08, October 26-31, 2008, Vanconver, British Columbia, Canada.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
97
Wang Shuo, Jing Feng, He Jibo, Du Qixing and Zhang Lei (2007) IGroup: Presenting Web
Image Search Results in Semantic Clusters, CHI 2007, April 28-May 3, 2007, San Jose,
California, USA.
White Ryen, W., Muresan Gheorghe, Marchionini Gary (2006) Evaluating Exploratory Search
systems, SIGIR 2006 Workshop, Seattle, USA.
White Ryen, W., Marchionini Gary, Muresan Gheorghe (2008) Evaluating Exploratory Search
Systems: Introduction to Special Topic Issue of Information Processing and Management,
Journal of Information Processing and Management- IPM, vol.44, no.2.
White Ryen, W., Dumais, S.T. and Teevan, J. (2009) Characterizing the influence of domain
expertise on web search behavior, Paper presented at the Second ACM International Conference
on Web Search and Data Mining, Barcelona, Spain.
Wilson, M. L. and Schraefel, M.C. (2008) Evaluating Collaborative Search Interfaces with
Information Seeking Theory, 1st International Collaborative Search Workshop, 20th June 2008,
Pittsburgh, PA, USA.
Wilson Max L., Kules Bill, Schraefel, M.C. and Shneiderman Ben (2010) From Keyword Search
to Exploration: Designing Future Search Interfaces for the Web, Foundations and Trends in Web
Science, Vol. 2, No. 1 (2010).
Zhang J. and Marchionini G. (2005) Evaluation and evolution of a browse and search interface:
Relation browser, Proceedings of the National Conference on Digital Government Research, pp.
179–188, Atlanta, GA, USA: Digital Government Research Center, 2005.
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
98
APPENDICES
APPENDIX 1
Survey Questionnaire
1. How often do you use the internet?
Daily Several times a week Weekly
Several times a month Monthly Never
2. Do you have any experience of exploratory search?
Yes No Preferred not to say
3. What is the main problem faced by you for exploratory search?
Unclear search goal
Unaware of how and where to find the information
Search system does not support better exploratory search
4. Are you aware of various exploratory search systems?
Yes No Preferred not to say
5. What do you use for exploratory search?
Search Engines Exploratory Search Systems Both
6. If you use search engine for exploratory search, how would you describe your ability to
use them?
Beginner Intermediate Experienced Expert
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
99
7. If you use exploratory search systems for exploratory search, how would you describe
your ability to use them?
Beginner Intermediate Experienced Expert
8. If you use Search engine, which one do you use most for exploratory search?
Google Bing Yahoo Live Search
Others
9. Among the following which type of approach do you prefer most for exploratory search?
Content-based search
Interface system
Visualization of search results
Query suggestions
10. How far existing Search engines or exploratory search systems support exploratory
search?
Bad ……….. 1
Fair ……….. 2
Good……… 3
Very Good... 4
Excellent….. 5
11. Are you aware of various techniques that will support exploratory search?
Yes No Preferred not to say
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
100
12. If answered yes to above question, which technique you think will support better
exploratory search?
Faceted Interface
Visualization
Clustering
Semantic web
Tags
APPENDIX 2
Evaluation Questionnaire
1 It was easy to use
the system
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
2 It was easy to learn
the use of system
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
3 I can effectively
complete task
using system
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
4 Efficiency of
system is good
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
101
5 Satisfied with
search results
provided by
system
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
6 Organization of
results is good and
useful
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
7 Satisfied with
information
acquired during
search
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
8 I am overall
satisfied with
system
1-Strongly
Disagree
2-Disagree 3-Neither
Disagree or
Agree
4-Agree 5-Strongly
Agree
COMPARISON OF GOOGLE AND DELICIOUS
Google Delicious
1 It was easy to use
the system
4.75 4.38
2 It was easy to learn
the use of system
4.63 4.25
3 I can effectively
complete task
using system
3.88 4.13
Evaluating Users Exploratory Information Seeking Behaviors
Madhusuthanan Nithyanandham
102
4 Efficiency of
system is good
4.06 3.94
5 Satisfied with
search results
provided by
system
3.81 4.0
6 Organization of
results is good and
useful
3.69 3.62
7 Satisfied with
information
acquired during
search
3.19 3.5
8 I am overall
satisfied with
system
3.93 3.81
APPENDIX 3
Interview Questions
1. What are the problems and difficulties faced by you when you are using the search systems for
exploratory search task?
2. What are the merits and limitations of the search systems?
3. What are the improvements needed for the current search systems?
4. What are the enhancements and features needed for the future development of search systems?
Top Related