Usage Statistics &

Information Behaviors: Understanding User Behavior with

Quantitative Indicators

John McDonaldAssistant Director for User Services &

Technology InnovationThe Libraries of the Claremont Colleges

November 2, 2007NISO Usage Data Forum

Correlation: Boba Fett and Ladybugs

We have the data, now what do we do?What we have done:

Cancel journals Inform purchase decisions

What we should do: Understand usage behaviors Guide our decision making processes Understand our impact on our patrons

Information Usage Behaviors

StartingBrowsingAccessingChainingDifferentiatingExtracting

Ellis (1993), Ellis & Haugan (1997) & Meho & Tibbo (2003), McDonald (2007)

VerifyingNetworkingMonitoringManagingManipulatingTeachingEnding

Accessing

Managing & Ending

Chaining & Differentiating

Accessing & BrowsingH

ow

Do W

e O

bserv

e

& M

easu

re t

hese

Beh

avio

rs?

How do we observe & measure? Pose a Question

How will a new service affect our users?

Develop a Theory Explain what you think happened.

Test the TheoryDevelop metrics, collect data, analyze.

Example 1: Starting & Accessing Question: How will a new service affect our users?

Theory: If we improve the user’s ability to identify relevant material (starting) and retrieve it (accessing), we either save them time or effort and allow them to access more material.

Test: There will be a significant increase in the usage of material.

Starting & Accessing: Use Before & After OpenURL

*significant at .05 level **significant at .01 level

Publisher Use 2000

Publisher Use 2001

Publisher Use 2002

Wilcoxon Signed-rank Test

Subjects Journals Mean SD Mean SD Mean SD z P>z

Astronomy 1 347 0 813 0 1408 0 -1.00 0.32

Biology 104 638 1625 847 2079 957 2351 -5.88 0.00**

Chemistry 42 1388 3248 1553 3889 2542 7294 -4.85 0.00**

Comp. Sci. 14 197 429 224 490 175 239 -1.63 0.10

Engineering 20 92 200 164 310 174 312 -2.41 0.02*

Gen. Sci. 3 16243 15571 20938 20345 26553 26506 -1.39 0.17

Geology 22 46 183 44 143 144 374 -3.10 0.00**

Mathematics 29 59 155 80 153 121 182 -3.68 0.00**

Physics 28 198 313 1081 2107 1526 2933 -4.00 0.00**

Total 263 701 2730 975 3527 1301 4953 -10.39 0.00**

Example 2: Differentiating

Question: Do our choices affect our users ability to differentiate between resources?

Theory: If we group resources together, we allow users to identify relevant resources and provide efficient methods to differentiate between resources.

Test: There will be a significant increase in searches across common resource groupings.

Differentiating: Federated Search Statistics

Database Searches

Web of Science 3823OPAC 3314WorldCat 3267PubMed 238INSPEC 233MathSciNet 183Faculty of 1000 Biology 176Compendex 132

Differentiating: OPAC Searches (2005 v. 2006)

Differentiating: WorldCat Searches

Example 3: Chaining

Question: Do our users move from one information resource to another?Theory: If users are moving from resource to resource, usage of resources in the same environment (one provider) and results of that usage (citations) will increase. Test: There will be a significant increase in the usage and/or results of usage of a resource’s material.

Chaining: JSTOR Citations (2000 v. 2004)

Example 4: Managing, Teaching

Question: Are our users managing or utilizing content differently?Theory: A stable online archive allows users to re-access or re-use content more efficiently (utility usage or virtual vertical file), or utilize it for instructional purposes in different ways (virtual syllabus). Test: There will be a significant increase in the systematic re-use of current, locally produced content.

Managing, Teaching: Use of local content

1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 20050

1

2

3

4

5

6

7

8

9

10Caltech Authored Articles v. Other Authored Articles

Example 5: Service Effects

Question: How do our choices in libraries affect user behavior?Theory: When we change the display options (e.g. cataloging) for journals, did that affect either publisher usage or SFX usage?

Test: Changing cataloging results in decreased local journal usage as measured by the publisher and SFX.

Service Effects: Usage of Journals (2005 v. 2006)

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec0

200

400

600

800

1000

Service Effects: SFX Clickthrough Rate (Local v. Shared)

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec50.0%

55.0%

60.0%

65.0%

70.0%

75.0%

80.0%

85.0%

90.0%

95.0%

Example 5: Services Related BehaviorsWhat else do users want or need?

Are there services related behaviors that we can observe? Providing content is one option, but how are researchers using associated information services? If we provide them the article they want in fulltext, we see that sometimes they ask for other types of things. Can we match these things to those user behaviors?

Services Related Behaviors

Information Service Requests

Order Article via Document Delivery 951

See References for this Article 790

Search Library Catalog 580

Read Abstract 283

Search Article Title on the Web 170

Send Feedback to Library 15

See Articles citing this Article 11

What else could we be studying?

Monitoring Many information providers have e-alerts, repeat saved searches, etc.

Networking Users may want to email a citation to a colleague or another student.

Extracting Passing the bibliographic information to another database to search.

Analyzing Including user behavior information in the statistical measurement tools.

Questions?

John McDonaldNovember 2, 2007