Information Seeking with Social Signals: Anatomy of a Social Tag-based Exploratory Search Browser

Information Seeking with Social Signals: Anatomy of aSocial Tag-based

Exploratory Search Browser

Ed H. Chi, Rowan Nairn

Palo Alto Research Center

Contact: [email protected] Manager, Augmented Social Cognition Area

2CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com

Social Search Survey

[Evans & Chi, CSCW2008]

150 user surveys Help understand the

importance of:– social cues and information

exchanges– vocabulary problems– distribution and

organization


TagSearch Exploratory Focus

3

3 kinds of search

navigational transactional

28% 13%

You know what you want and where it is You know what you want to do

Existing search engines are OK

informational

59%

You roughly know what you want

but don’t know how to find it

Difficult for existing search engines

Opportunity


Research Motivation


MapReduce Implementation

Spreading Activation in a bigraph MapReduce computation over a large data set

– 150 Million+ bookmarks

Tags URLs

P(URL|Tag)

P(Tag|URL)


Guide

Web

Howto

TipsHelp

Tools

Tip

Tricks

Tutorial

Tutorials

Reference

Semantic Similarity Graph

Use Semantic Analysis to Reduce Noise

8CHI2009 MrTaggy TagSEarch– © 2008 Palo Alto Research Center Inc.

TagSearch Architecture

• MapReduce: months of computation to a single day• Development of novel scoring function


Baseline Interface


Exploratory Interface


Experiment Design

2 interface x 3 task domain design– 2 Interface (between-subjects)

» Exploratory vs. Baseline– 3 task domains (within-subjects)

» Future Architecture, Global Warming, Web Mashups

30 Subjects (22 male, 8 female)– Intermediate or advanced computer and web search skills– Half assigned Exploratory, half Baseline.

For each domain, single block with 3 task types:– Easy and Difficult Page Collection Task [6min each]– Summarization Task [12min]– Keyword Generation Task [2min]


Page Collection Tasks [6min each]


Summarization Tasks [12min each]


Procedure [2 hours]

Prior Knowledge Test 1st Task Domain

– With easy and difficult page collection tasks, summarization and keyword generation task.

– NASA cognitive load questionnaire

2nd Task Domain– Same battery of tasks and cognitive load questionaire

3rd Task Domain Experimental Survey


Results: Interaction Behaviors

Number of Queries– Effect of Interface on number of queries (p < .01)

» Exploratory (M=7.81) > Baseline (M=3.77)

Time Taken– Effect of Interface on time taken (p < .01)

» Exploratory (7.7min) > Baseline (6.6min)


Results: Page Collection Task

0

2

4

6

8

10

12

14

16

task 1 task 2 task 1 task 2 task 1 task 2

global warming future architecture web mashups

nu

mb

er

of

pag

es c

olle

cte

d

Baseline Exploratory

– Effects of Task Domain (p<.01) and Task Difficulty (p<.05)

– Interaction effect of Interface by Task Domain (p<.05), with Exploratory interface performing better in the Web Mashup domain

– For relevance scores, similar patterns.

Measure of # of pages collected


Results: Summarization Tasks

0

1

2

3

4

5

futurearchitecture globalwarming mashups

nu

mb

er

of

sum

mar

y a

sp

ect

s

baseline exploratory

– Quality of summarization scored (Cohen’s Kappa=0.7)

– ANCOVA with Prior Knowledge as covariate

– Exploratory Interface scored higher in Future Architecture (p<.05) and Global Warming (p<.05)

– For Web Mashup, Prior Knowledge correlated positively with performance (r=.51)


Results: Keyword Generation Tasks

0

2

4

6

8

10

12

14

16

18

globalwarming futurearchitecture mashups

nu

mb

er o

f ke

yw

ord

s


– ANCOVA showed Exploratory > Baseline for Future Architecture (p<.05) and Web Mashups (p<.01), but not for Global Warming.

– Linear model between PK and # of keyword generated for Baseline showed mean slope = 0.32 and significant (p<.05)


Results: Cognitive Load

0

20

40

60

80

100

futurearchitecture globalwarming mashups

co

gn

itiv

e l

oa

d


– Exploratory > Baseline (p<.05)


Discussion

Exploratory interface users:– performed more queries, – took more time, – wrote better summaries (in 2/3 domains), – generated more relevant keywords (in 2/3 domains), and– had a higher cognitive load.

Suggestive of deeper engagement and better learning.

Some evidence of scaffolding for novices in the keyword generation and summarization tasks.


Summary

Harnessing user-generated tags to enrich content for social search

Weaknesses of social tagging systems is Tag Noise and Inconsistency– Difficult to leverage for search– Use data mining techniques to normalize and reduce noise– Apply normalized tag data in new search algorithm

Study suggest deeper user engagement in exploration and better learning with MrTaggy


Thanks!

http://mrtaggy.com

http://spartag.ushttp://wikidashboard.parc.com

Our Blog: http://asc-parc.blogspot.com

Contact:Ed H. Chi, Ph.D.Manager, Augmented Social Cognition [email protected]

Kammerer, Y., Nairn, R., Pirolli, P., and Chi, E. H. 2009. Signpost from the masses: learning effects in an exploratory social tag search browser. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI '09. ACM, New York, NY, 625-634.

http://mrtaggy.com/

http://spartag.us/

http://wikidashboard.parc.com/

http://asc-parc.blogspot.com/

mailto:[email protected]

Research VisionResearch Vision

Augmented Social CognitionAugmented Social Cognition Cognition: the ability to remember, think, and reason; the

faculty of knowing. Social Cognition: the ability of a group to remember, think,

and reason; the construction of knowledge structures by a group.– (not quite the same as in the branch of psychology that studies

the cognitive processes involved in social interaction, though included)

Augmented Social Cognition: Supported by systems, the enhancement of the ability of a group to remember, think, and reason; the system-supported construction of knowledge structures by a group.

Citation: Ed H. Chi. The Social Web: Opportunities for Research. IEEE Computer, Sept 2008

232008-11-07 Ed H. Chi ASC Overview

Collective Intelligence

Augmented Social CognitionAugmented Social Cognition

24

Higher Productivity via Collective Intelligence

Intelligence that emerges from the collaboration and competition of many individuals

search

sharing

foraging

TagSearch: Mining social data for automatic data clustering and organization:

• Better organization via user-assigned tags

• Better UI for browsing interesting contents

• Recommendation instead of just search

Social Transparency create trust and attribution:

• Increase participation via attribution

• Increase credibility and trust with community feedback

• Reduce wiki risks

SparTag.us: sharing of interesting contents:

• A notebook that automatically organizes your reading

• Social sharing of important and interesting tidbits

• Viral sharing of highlighted and tagged paragraphs

Foundation:• Understanding of human

cognition and behavior• Data mining of social data

Generic benefits:• Greater trust• Better decision-making• Useful sharing of info• Auto-organization thru

social data

Information Seeking with Social Signals: Anatomy of a Social Tag-based Exploratory Search Browser

Technology

Transcript of Information Seeking with Social Signals: Anatomy of a Social Tag-based Exploratory Search Browser