IOTA: Improving OpenURL Through Analytics

Nettie Lagace (@abugseye)NISO Associate Director for Programs

CEAL Workshop on Electronic Resources Standards and Best Practices

March 25, 2014

Thanks to Oliver Pesch of EBSCO for much of this slide material.

• Simple example assuming equal element weightsElement Description Weight This OpenURL

ATitle Article title 1

AuLast Author’s last name 1

Date Date of publication 1

ISSN ISSN 1

Issue Issue number 1

SPage Start page 1

Title Journal Title 1

Volume Volume number 1

TOTAL 8

SAMPLE OPEN URL DATA ?date=2008-02-04 &issn=1083-3013 &volume=13 &issue=20 &atitle=the+casualties+of+war

1

1

1

1

1

5

Completeness Score...

(Total for This OpenURL)Total Weights

5 / 8

= .625

IOTA Recommended Practice

• Defines a technique for determining element weights• Tested with real link resolvers and real OpenURLs• Based on research which looked for a correlation with

data elements on the OpenURL and “success” of the OpenURL

• Select a set of “perfect” OpenURLs – include all key data elements and resolve to full text

• Perform step-wise regression– Test failure rates for each element by removing that

element• Use failure rates as basis for weights• Use weights to calculate Completeness Scores

and to test for correlation between weights and success for larger sample

A Statistical Approach to Determining Element Weights

Element removed from the OpenURL

Description Failure Percentage

ATitle Article title .74%AuLast Author’s last name .07%Date Date of publication .4%ISSN ISSN (either online or

print ISSN)22.02%

Issue Issue number 20.27%SPage Start page 33.27%Title Journal Title (either

Title or Jtitle).61%

Volume Volume number 74.14%

Volume is most critical

Author’s last name is least important

Date is surprisingly low

Failure Rates from 1500 OpenURL test sample

Element Description Weight*

ATitle Article title 1.87

AuLast Author’s last name 0.83

Date Date of publication 1.61

ISSN ISSN (either online or print ISSN)

3.34

Issue Issue number 3.31

SPage Start page 3.52

Title Journal Title (either Title or Jtitle)

1.78

Volume Volume number 3.87

*Element weight calculation: log10 (failure-rate-per-10,000 OpenURLs)

Calculated Element Weights

Correlation Coefficient .80Tests conducted on sample of 15,000 OpenURLs randomly pulled from IOTA database

0.00000.20000.40000.60000.80001.00001.2000

Average of Completeness Score

Average of Success Score

Results

IOTA Online Tool

• 23.3+ million OpenURLs processed• Reporting interface– Analyze data elements (metrics) across vendors or database

(Source)– Analyze (Source) for all data elements

Analysis of vendors by element (metric)

Analysis of elements by vendor

How to use IOTA materials

• The Technical Report provides suggestions for improving OpenURLs

• The interactive tool offers a means to pin-point irregularities in data provided on OpenURLs

• The Recommended Practice describes how to create a Completeness Index

• Completeness Index allows OpenURL quality problems to be quantified

Why you should use IOTA

• Link resolver vendors can implement the Completeness Index in their products to help identify problematic OpenURL sources

• Librarians can use suggestions and Completeness Index to more effectively communicate quality problems to content providers

• Content providers can use the online interactive tool to identify problems with the data they provide

Thank you! Questions?nlagace@niso.org

@abugseye

I Want To Go To Magnolia Mountain by lilomooney is licensed under CC-BY 2.0