Post on 29-Dec-2015
User studiesUser studies
LIS 670Bair-Mundy
Users
Potential Potential usersusers
ExpectedExpectedusersusers
ActualActualusersusers
BeneficiaryBeneficiaryusersusers
Studying users
What we need to knowWhat we need to know
Types of user studiesTypes of user studies
Designing a research planDesigning a research plan
Problems in user researchProblems in user research
User study assignmentUser study assignment
Data collection methodsData collection methods
What we need to know (1)
Information needs of users
Request for Assistance
Name___________
Address_________
Telephone_______
What we need to know (2)
How users utilize information
What we need to know (3)
How the user attempts to fulfillhis/her information needs
Information-seeking activities
User reasoning processes
What we need to know (4)
Means of presentation of information that are intelligibleand thus useful to the user
What we need to know (5)
Trustworthy sources of information for the user
What we need to know (6)
Elements of a user interface that expedite or hinder information retrieval
Enter search statement in the box below
START CAT FIND XT GONE WITH THE WIND
Intelligible and useful means of presentation of information
Searching by color
http://portia.nesl.edu/screens/well_its_red.html
Types of user studies
UserUserorientedoriented
Systems-useSystems-useorientedoriented
UtilityUtilityorientedoriented
User-oriented studies
Seek to utilize demographics to predict information use
Age
Education level
Economic status
Language
Demographics - language
http://www.lapl.org/espanol/index.html
Systems-use oriented studies
Extent to which users utilize one or more kinds of information system
Degree of difficulty or satisfaction in using information services
Characteristics of the system contributing to success of
information transactions
Elements examined in systems-use oriented studies (1)
Number & type of reference questions asked
Number of items loaned in-house
Number of items accessed through interlibrary loan
Elements examined in systems-use oriented studies (2)
Number of times users access online catalog from remote sites
Search and retrieval statistics for online databases
Number of times users access particular online databases
Ethical consideration in systems-use studies: user privacy
Aggregate data about systems-use is acceptable
Keep only data necessary for providing services for only as long as needed
Remove any information that could identify individual
Utility-oriented studies
Critical incidence studiesSubjects report on information needsand their resolution at each decision-making point in a specific search
Citation analysisAnalyze citations to articles by particularauthor or published in a particular journal
Process of user research
1. Design of research plan
2. Implementation of research plan
3. Implementation of results
4. Evaluation of results
Designing a research plan:Step 1
Identify a problem area orneed to study
Not as easy as it sounds!
Problem: Line at the reference desk?
Reference
Actual problem: lack of signage
ReferenceAdult Fiction
Children's Room
Reference Desk
Restrooms
Science & Tech
Designing a research plan:Step 2
Conduct an initialliterature review
How have others approached the problem?
Designing a research plan:Step 3
Define the specificresearch question
to study
What is the optimal arrangement of information resources in the reference area?
Designing a research plan:Step 4
Cost-benefit analysisof study
Study may be expensive. Will resulting suggestions be implemented?
Designing a research plan:Step 5
Conduct a secondliterature review
Specific search now that research problem is defined
Designing a research plan:Step 6
Formulate hypotheses
(specific testable question(s) to
ask in the study)
"Changing the configuration of the furniture to segregate incoming and outgoing traffic will result in less crowding in the lobby."
Designing a research plan:Step 7
Identify the populationto study
Everyone in certain zip codes?
Designing a research plan:Step 8
Selectdata collection
methods
Phone survey? In-house questionnaire?Capture keystroke data?
Designing a research plan:Step 9
Developdata collection
instruments
Questions for phone survey
or questionnaire
http://www.isworld.org/surveyinstruments/surveyinstruments.htm
From 1 (strongly disagree) to 6 (strongly agree), rate the following statements by completing this sentence: From the courses I took in the LIS Program, I feel I gained sufficient knowledge of the ...
Rating (circle one)
2.1) Philosophy, principles, and ethics of library and information science
1 2 3 4 5 6
2.2) Historical development and communication of knowledge
1 2 3 4 5 6
2.3) Use of strategic planning techniques to create short- and long-term plans for improvement
1 2 3 4 5 6
…
Strongly disagree
Strongly agree
Survey of currently-enrolled LIS students
A continuación puntúe de 1 (muy deficiente) a 6 (muy satisfactorio) las siguientes variables
Grado de conocimiento que usted tiene sobre los servicios que ofrece la biblioteca
1 2 3 4 5 6Localización de la biblioteca 1 2 3 4 5 6Indicación de su ubicación1 2 3 4 5 6Infraestructura de la sala1 2 3 4 5 6Tamaño de la biblioteca1 2 3 4 5 6...
Surveying a multi-ethnic community
Designing a research plan:Step 10
Design adata-analysis
plan
Statistical tests you plan to use will influence way you collect and encode
data.
Salary dataStatistical tests you plan to use will influence way you collect and encode data.
What is the starting salary for librarians at your institution?
What is the starting salary for librarians at your institution?
$20,000 - $29,000 $60,000 – $69,000
$30,000 - $39,000 $70,000 – $79,000
$40,000 - $49,000 $80,000 – $89,000
$50,000 - $59,000 $90,000 – $99,000
Designing a research plan:Step 11
Formulate adata-collection
plan
Stopping people who walk in the door to
question them?
Designing a research plan:Step 12
Conduct pilot studies ofmethods, instruments,
and analysis
First try questionnaire on friends and staff, then on sample from group you plan to study
Data collection methods
QuestioningQuestioning ObservingObserving
StudyingStudyingrecords orrecords ordocumentsdocuments
Questioning (1)
Questionnaire survey—can reach widely dispersed geographic group of subjects; low return rate; low validity
Interview—can give high quality data due to follow-up questions; costly; limited sample size; inconsistency among interviewers
Questioning (2)
Diary method—rich informational content; low validity and reliability (answers get shorter over time)
Group interview and questions—less expensive than individual interview; good response rate; includes Delphi technique
Delphi technique
Summarize responses and distribute to the group
Ask for revisions of original opinions based on group responseReport group response to individuals again
Elicit new revisions of opinions
Ask each expert same questions
Observation
Can be less intrusive
Can be highly reliable
Objective
Expensive
Doesn’t reflect motivation
Hawthorne effect—activities of observed subjects are affected by interested presence of the observer
Hawthorne effect
Lighting
productivity
Lighting
productivity
Documentary evidence
PublicationsPublications StatisticalStatisticalreportsreports
CitationsCitations
Publications
Information Science Research
Library Science Research
Bibliometrics
Citations
Citations
Statistical report - databasesCAMBRIDGE SCIENTIFIC ABSTRACTS, 2002
Database Name J an Feb Mar Apr May J unPsycINFO (1984-Current) 1855 3203 3558 4500 2078 2487ERIC 1722 2076 1914 2871 651 872ASFA: Aquatic Sciences and Fisheries Abstracts 1330 1530 2117 2156 914 1021AGRICOLA 1115 1318 1348 1681 677 662Linguistics and Language Behavior Abstracts 475 932 1060 959 514 352Oceanic Abstracts 563 961 1196 1383 372 642Sociological Abstracts 456 687 664 742 327 436EconLit 387 437 844 801 617 236PsycINFO (1967-1983) 320 707 1071 1039 967 698Zoological Record Plus (1978-Current) 0 0 0 668 284 99PsycINFO (1887-1966) 113 391 593 447 695 489Social Services Abstracts 94 181 241 306 108 180Meteorological & Geoastrophysical Abstracts 0 0 0 0 0 0Zoological Record Plus (2001) 531 521 428 73 36 5Zoological Record Plus (2000) 503 491 391 73 36 5Zoological Record Plus (1997-1999) 500 491 349 41 35 5Zoological Record Plus (1993-1996) 453 430 280 41 35 1Zoological Record Plus (1985-1992) 408 367 258 29 31 1Zoological Record Plus (1978-1984) 351 337 245 28 31 1BioEngineering Abstracts 244 214 135 304 76 0Environmental Engineering Abstracts 101 56 124 389 115 0ARTbibliographies Modern 154 101 79 42 22 5
No. of Times Each Database Was Queried in 2002
Statistical report - searchesUniversity of Hawaii at Manoa LibraryLexisNexis Usage Report, 2002-03
Documents Documents Documents DocumentsSearches Retrieved Searches Retrieved Searches Retrieved Searches Retrieved
Jun-98 457 694 1 2 66 82 524 778Jul-98 736 1,182 0 0 52 75 788 1,257Aug-98 1,489 3,047 11 20 69 79 1,569 3,146Sep-98 1,346 2,230 85 96 77 98 1,508 2,424Oct-98 1,377 3,470 48 85 148 210 1,573 3,765Nov-98 1,280 2,756 20 28 40 45 1,340 2,829Dec-98 1,521 3,017 167 196 25 39 1,713 3,252Jan-99 2,437 4,737 68 76 68 76 2,573 4,889Feb-99 1,956 4,497 109 206 90 113 2,155 4,816Mar-99 3,350 6,101 65 65 47 56 3,462 6,222Apr-99 1,429 2,734 59 101 22 29 1,510 2,864May-99 1,376 2,532 22 18 27 19 1,425 2,569
Total 18,754 36,997 655 893 731 921 20,140 38,811
Academic Universe Congressional Universe Statistical Universe All Three Databases
Directional Hawaiian Pacific Rare
Reference question tally
OPAC Logging Report
System-use printout
DATE TYPE SEARCH STRING LIMIT LIMIT_STRING INDEX HITS9/28/01 Keyword harry potter N K 4068
9/28/01 Author tolkein N LOCA=Main Library B 1
9/28/01 Keyword birds N K 2863
9/28/01 Keyword alligators N LOCA=Main Library K 30
9/28/01 Complex (NKEY alligators) AND N K 0
(TKEY "crocodiles")
9/28/01 Complex (TKEY rats) AND N K 103
(TKEY "mice")
9/28/01 Keyword harry potter goblet N K 6848
Classical experiment
Formulate hypothesis
Experimental group is tested in comparison with control group
Must keep all other variables constant (difficult to do with human subjects)
Different reactions
Welcome Welcome
excellent poor
Subject A Subject B
Problems in user research (1)
Many variables affect information-seeking behavior
Variables are difficult to identify
Variables are difficult to control
Problems in user research (2)
Link between cause and effect difficult to establish
Quality of service not readily measurable
NoticeChange of borrowing policy to take effect
?circulation
X
Users’ articulated demands may not reflect their true information needs
Request:
Articles about New Zealand in National Geographic
Need:
Information about business opportunities in New Zealand
Studies are often disruptive to normal operation of information-provision system
Reference
Difficulty in implementing changes suggested by result of studies
Reference.
.
..
.
.
.
.
.
..
Factors that affect qualityof data
ValidityIs data representative of total
population?Does measuring tool actually yield
desired information?
ReliabilityStability of the measure over time
(e.g. interviewer consistent with every subject?)
In-class exercise: User study
Read scenario
Analyze the problem
Decide on research methodology
Identify population to study
Select data collection method(s)
Three sample questions
Formulate a data-collection plan
Assignment:
Recommendation for a User Study
Read scenario
Analyze the problem
Decide on research methodology
Write letter describing & justifying methods