Recommendations from Readers' Choices in Library Networks...

Recommendations from Readers’ Choices:An Application of Small Sample Statistics

–An Insight into Current Library Research

Dr. Andreas W. Neumann

Information Services and Electronic MarketsInstitute of Information Systems and Management

Department of Economics and Business Engineering

Universitat Karlsruhe (TH)

Lecture ”Die Digitale Bibliothek“July 2, 2008, Karlsruhe, Germany

DR. ANDREAS W. NEUMANN

Table of Contents

• Research projects at the Universitat Karlsruhe (TH)

• Introduction and motivation of recommender systems

• Application area and statistical properties of the data

• First method: Probability of single item co-inspections (POSICI)

• Second method: Probability of multiple items co-inspections (POMICI)

• Evaluation, interpretation and comparison

• Further Research

DR. ANDREAS W. NEUMANN Recommendations from Readers’ Choices: An Application of Small Sample Statistics 1

Project Overview

Joint Projects of the Institute of Information Systems and Management togetherwith the University Library at the Universitat Karlsruhe (TH) funded by theDeutsche Forschungsgemeinschaft (DFG):

• 2002/02 - 2004/01: Scientific Libraries in Information Markets

– Strategic positioning options for scientific libraries– Improving the competitive position of scientific libraries in information markets– Project homepage: http://wbii.em.uni-karlsruhe.de/

• Since 2004/07: Recommender Systems for Meta Library Catalogs

– Permanent improvement of the information infrastructure of scientific librariesby developing customer oriented service portals

– Project homepage: http://reckvk.em.uni-karlsruhe.de/

• Since 2006: BipTip http://www.bibtip.org/

• At large: How to make our everyday scientific work and studies moreproductive?


http://wbii.em.uni-karlsruhe.de/

http://reckvk.em.uni-karlsruhe.de/

http://www.bibtip.org/

Recommender Systems for Scientific Libraries – Why?

• Scientists and students are more and more incapable of efficiently findingrelevant literature in conventional database oriented catalog systems andsearch engines.

• Problems: Supply complexity, evaluation of the quality, information overload

• Typical literature research path: Asking peers

• Recommender systems aggregate knowledge from many peer groups to thelevel of expert advice services.

• Recommender systems bear the potential to significantly reduce transactioncosts for literature searches by means of their aggregation capabilities.

• Library OPACs can be turned into customer oriented service portals supportingthe interaction of the customers.


Behavior-Based Recommender Systems

Recommender System. Reads observed user behavior or opinions from usersas input, then aggregates and directs the resulting recommendations toappropriate recipients.

Behavior-Based Recommender System. Based on behavioral usage data:Others also use . . .

We test our methods on usage data from scientific libraries. The methods aregenerally applicable in any consumer store setting.

• Data: Anonymized library OPAC searches (hits on document inspection pages= purchases)

• Session analysis (market baskets): Together with this document people alsolooked at (bought). . .

• Problem: Which co-inspections (co-purchases) occur random, which not?


User Interface: Library OPAC Search


User Interface: Hit List


User Interface: Document Inspection Page

Detailed view of documents (books, journals, multi-media,. . . )


User Interface: List of Recommendations


Statistical Properties of the Data (Market Baskets)Status of 2007-02-19

UBKA KVKNumber of total documents in catalog 1,000,000 > 10,000,000Number of total co-inspected documents 527,363 255,248Average market baskets size 4.9 2.9Av. aggregated co-inspections per document 117.4 5.4

• When starting to monitor new catalogs (stores) no information about the usageof the documents (products) in the catalog is available.

• With smaller market baskets and a greater number of total documents it takesmuch more time for many co-inspections per document to occur.

• Due to sample size constraints methods using statistical tests on distributions(like LSD) are only reliably applicable with many co-inspections.

• Special small sample statistics are needed to compute recommendations out ofsamples of few co-inspections.


Distributions of Co-Inspections (KVK Data)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Distribution of Co−Inspections for k <= 15

Number of Co−Inspections

Num

ber

of D

ocum

ents

020

000

4000

060

000

8000

010

0000


Which Co-Purchases Occur Non-Random?

• One Solution (large samples): Stochastic purchase incidence model based onEhrenberg’s repeat-buying theory (1988), descriptive theory of consumerbehavior

– The frequency distribution of library document co-inspections (co-purchases)is following a logarithmic series distribution (LSD)

– Operational at the university library of Karlsruhe since◦ Catalog of the university of Karlsruhe (UBKA): June 2002◦ Union catalog of Karlsruhe’s public libraries (KGK): November 2005◦ Karlsruhe’s Virtual Catalog (KVK) (Meta-Catalog searching 52 international

catalogs): March 2006

• New Solutions (small samples):

– POSICI: Probability of single item co-inspections– POMICI: Probability of multiple items co-inspections


Data Setup

• The number of total documents n+1 in the catalog is finite but unknown.

• Recommendations are computed separately for each document D.

• Each user session (market basket) contains all documents that the userinspected within that session, multiple inspections of the same document arecounted as one.

• All user session are aggregated.

• The set C(D) contains all documents, that at least one user has inspectedtogether with D.

• The number of co-inspections with D of all elements of C(D) is known, thishistogram is called H(D).

• H(D) can be intepreted as an integer partition with the number ofco-inspections of each co-inspected document as the summands.


General Concept

Assumption: All documents in the catalog have the same probability of beingco-inspected.

• In real systems generally this assumption does not hold.

• Especially when starting to observe new catalogs no information about theunderlying distribution of the inspection processes of documents is known.

• We do not develop a causal model of a decision maker, we construct an idealdecision maker without preferences.

General Concept: Recommendations are co-inspections that occur significantlymore often then predicted in the case of the assumption being true.


An Ideal Decision Maker

• Is a prototype for a class of homogenous decision-makers

• Makes choices randomly. But from what choice set?

• The conceptual model of the choice set:

Total set ⊇ awareness set ⊇ consideration set ⊇ choice set (Kotler 1980)

• Evidence for the model:

– Awareness set from 3 – 11 products (Narayana and Markin (1975)– Interaction of emotional and rational brain activity (Bechara et al. 1997):

Emotional activity reduces size of choice set of normal persons, whereas forpatients with prefrontal damage this could not be observed.

– In a brand choice experiment successful branding reduces the choice set to 1(the winning brand) and the regions of the brain responsible for emotionsshow high activity when measured by fMRI (Deppe et al. 2005).

Consequence: We consider a sequence of increasing event spaces.


Setup Overview and Example

• To ease the cold start problem of new recommender services

• Noise: ideal decision makers without preferences(all documents have the same probability of being co-inspected)

• Unknown total number of documents n+1 in the meta-catalog

• Aggregated co-inspections are written as integer partitions(Euler, MacMahon, Hardy, Ramanujan, Rademacher)

• Example: 4+1+1 co-inspections with document D

– Total of k = 6 (sum) co-inspections– Co-inspected with 3 (number of addends) other documents◦ 4 times with document X◦ 1 time with document Y◦ 1 time with document Z

• Algorithm input:{

[4] , [1] ⇒ POSICI[4+1+1] ⇒ POMICI


POSICI: Probability of Single Item Co-Inspections

Question: What is the probability p j that at least one other document has beenco-inspected exactly j-times with document D?

• Let n+1 be the number of total documents, k the number of non-aggregatedco-inspections (multiple inspections in different sessions are countedseparately)

• Let (N1, . . . ,Nn) be the vector of the number of times document i (1≤ i≤ n) wasco-inspected with D

• Then (N1, . . . ,Nn)∼M (k; p1, . . . , pn) , pi = 1n, 1≤ i≤ n

• Let Ai = {Ni = j}

Answer: (inclusion-exclusion principle)

p j = P

(n⋃

i=1

Ai

)=

n

∑ν=1

(−1)ν−1∑

1≤i1<...<iν≤nP(Ai1∩ . . .∩Aiν)


Inspection Probabilities for Growing k and n in POSICI

10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0

Co−inspection probabilities for k = 5 (n = 5 to 50)

n

prob

abili

ty

P(1)P(2)P(3)P(4)P(5)


POSICI Recommendation Generating Algorithm

1. Let D be the document for which recommendations are calculated.

2. Let n = k and t a fixed chosen acceptance threshold (0 < t < 1).

3. Calculate p j for j = 1, . . . ,k.

4. For all with D co-inspected documents C do

(a) If document C has been co-inspectioned j-times and p j < t p1

then recommend C.


Inspection Probabilities for Growing k and n in POSICI

10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0

Co−inspection probabilities for k = 8 (n = 8 to 50)

n

prob

abili

ty

P(1)P(2)P(3)P(4)P(5)P(6)P(7)P(8)


Interpretation of POSICI

Example for k = 9:

• Observed partitions:

– (1) 5+2+2– (2) 5+2+1+1

• The 5-times co-inspected document has the same probability to be arecommendation in both cases.

POSICI is built on the theory, that co-inspections other than j-times add morenoise than information about the incentive to co-inspect the current document

j-times.


POMICI: Probability of multiple items co-inspections

Question: What is the probability ppart that the partition corresponding to thecomplete histogram H(D) of all co-inspections with D occur?

• Let n+1 be the number of total documents, k the number of non-aggregatedco-inspections (multiple inspections in different sessions are countedseparately)

• Let X be the set of words of length k from an alphabet of n letters, and li thenumber of letters (i. e. documents), that occur exactly i-times in x ∈ X (i. e. inH(D))

• First examine the actions of the group G = Sn×Sk on the set X , and then theactions of the stabilizer subgroup Gx on the set Sn for the identitiy id ∈ Sn

Answer: (applying two times the orbit-stabilizer theorem together with Lagrange’stheorem from group theory and some counting arguments)

ppart =|Gx||X |

=|G||Gx|

=|G|

|Gxid| |Gxid|=

n! k!(n−∑

ki=1 li

)! ∏

kj=1 l j! ( j!)l j


Partition Probabilities for Growing k and n in POMICI

10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0

Partition probabilities for k = 3 (n = 3 to 50)

n

prob

abili

ty

1+1+12+13



10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0


n

prob

abili

ty

1+1+1+12+1+12+23+14



10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0


n

prob

abili

ty

1+1+1+1+12+1+1+12+2+13+1+13+24+15



10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0


n

prob

abili

ty

1+1+1+1+1+12+1+1+1+12+2+1+12+2+23+1+1+13+2+13+34+1+14+25+16



10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0


n

prob

abili

ty

1+1+1+1+1+1+12+1+1+1+1+12+2+1+1+12+2+2+13+1+1+1+13+2+1+13+2+23+3+14+1+1+14+2+14+35+1+15+26+17



10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0


n

prob

abili

ty

1+1+1+1+1+1+1+12+1+1+1+1+1+12+2+1+1+1+12+2+2+1+12+2+2+23+1+1+1+1+13+2+1+1+13+2+2+13+3+1+13+3+24+1+1+1+1

4+2+1+14+2+24+3+14+45+1+1+15+2+15+36+1+16+27+18


POMICI Recommendation Generating Algorithm

1. Let D be the document for which recommendations are calculated.

2. Let t a fixed chosen acceptance threshold (0 < t < 1).

3. Let nD be the smallest integer, after which the order by probability of thepartitions for n≥ nD is constant.

4. At nD do

(a) Let s be the largest integer that occurs in the partition with the highestprobability below t p1+...+1.

(b) For all partitions part with ppart < t p1+...+1 doi. Recommend all documents from H(D) that have been co-inspected at least

s-times.



10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0


n

prob

abili

ty

1+1+1+1+1+12+1+1+1+12+2+1+12+2+23+1+1+13+2+13+34+1+14+25+16


Interpretation of POMICI

Example for k = 9:

• Observed partitions:

– (1) 5+2+2– (2) 5+2+1+1

• The 5-times co-inspected document has not the same probability to be arecommendation in both cases.

• Partition (2) is more likely, so here the 5-times co-inspected document has avery slightly higher chance of being recommended.

POMICI is built on the theory, that the distribution of co-inspections other thanj-times reveals more information than noise about the incentive to co-inspect the

current document j-times.


POSICI vs. POMICI

• POSICI and POMICI are related

– POSICI is POMICI aggregated over partitions– POMICI is a refinement of the multinomial model in POSICI– The sum of the probabilities of all partitions from POMICI with at least one

product that was co-inspected exactly j-times is equal to the probability inPOSICI, that there exists at least one product, that was co-inspected exactlyj-times.

By setting the threshold t for POSICIand POMICI respectively, the numberof generated recommendations can beadjusted for both methods.

POMICI generally generates longerrecommendation lists for fewer docu-ments than POSICI.

POMICI(t=0.02)

POMICI(t=0.05)

POMICI(t=0.1)

POSICI(t=0.1)

POSICI(t=0.2)

POSICI(t=0.3)

POSICI(t=0.4)

POSICI(t=0.5)

POMICI vs. POSICI

Method

Num

ber

020

040

060

080

010

0012

00

Documents with RecommendationsTotal Recommendations


Further Research

• When does the partition tail of smaller integers resembles noise and whenincentive behavior?

• Human evaluation of larger data sets has to determine if POSICI or POMICIleads to qualitatively better recommendations in a specific setting.

• If the overall inspection probability of documents is known (through largebehavior data sets), both methods can be extended to include theseprobabilities as weights to improve the recommendations.

• Portraying the additions of further co-inspections (k → k +1) as aMarkov-process to calculate the probability of a product with currently lowco-inspections to develop into high co-inspections, thus a reliablerecommendation.


Thank you for your attention.The author gratefully acknowledges the funding of the project“Recommender Systems for Meta Library Catalogs” by theDeutsche Forschungsgemeinschaft.

• Reference: Andreas W. Neumann and Andreas Geyer-Schulz, “Applying SmallSample Test Statistics for Behavior-Based Recommendations”. In DataAnalysis, Machine Learning, and Applications, Springer, Berlin Heidelberg,2008.

• Recommender systems available at:http://www.ubka.uni-karlsruhe.de/

• Usage information:http://reckvk.em.uni-karlsruhe.de/participate eng.php

Dr. Andreas W. [email protected]

http://www.em.uni-karlsruhe.de/home/ane/


http://www.ubka.uni-karlsruhe.de

http://reckvk.em.uni-karlsruhe.de/participate_eng.php

http://www.em.uni-karlsruhe.de/home/ane/

Recommendations from Readers' Choices in Library Networks...

Documents

Transcript of Recommendations from Readers' Choices in Library Networks...