Case Based Reasoning

Lecture 7: CBR Competence of Case-Bases

Outline

The Utility Problem & Case Deletion A First Model of Case Competence

Case Competence Categories Competence-Preserving Deletion

A Second Model of Case Competence Competence Groups

Reading

Case-Base Maintenance

Case redundancy duplicates or unnecessary near neighbours in case-base

may evolve during retain redundant cases may not harm decision making, but can

slow down the system consult domain experts

are potentially redundant cases harming performance? or may they be useful in future?

Case utilisation statistics how many times is each case retrieved? if case never retrieved over a period of time

may be redundant if case retrieved very frequently

may indicate poor case coverage

The Utility Problem

The utility problem occurs when cost of searching for relevant knowledge

outweighs benefit of applying knowledge In CBR large case-bases mean expensive

retrieval To cope with CBR utility problem

delete any cases that do not affect the competence to solve problems the performance (time)

i.e. lean case-bases

A First Case Competence Model

All cases are not equal Case Competence Categories

Pivotal cases contribute to competence Auxiliary cases contribute to performance

Intermediate categories Spanning cases Support cases

Competence-Preserving Deletion Categorise cases Order for deletion in terms of contribution to

competenceSmyth & Keane

Case Competence – The Basics

Ideal measure of case coverage the set of target problems that it solves

For a case c and a target problem t solves(c,t) means c solves t

c is retrieved for t c can be adapted to solve t

For a case c and a target problem set T coverage(c)={tT : solves(c,t)}

Infeasible to generate set of all targets T space of target problems is too vast

coverage( ) = { s}

Target

Case

Case Competence – The Basics

Practical Measure of Case Coverage the set of cases in the case-base that it solves assumes case-base C is a representative

sample of T coverage(c)={c′ C : solves(c,c′)}

the set of cases in the case-base C that c is retrieved for c can be adapted to solve

Case

Case

coverage( ) = { s}

Case Competence – The Basics

Reachability of a target problem t set of cases in C that provide a solution for it reachability(t) = {c C : solves(c,t)}

Interested in reachability(c) for c C

reachability(c)={c′ C : solves(c′,c)}

reachability( ) = { s}

Case

Case

Competence Categories: Pivotal

A case is pivotal if it is reachable by no other case but itself pivotal (c) iff reachable(c) = {c}

Pivotal cases are generally outliers too isolated to be solved by any other case

Target problems falling within the region of a pivot can be solved only by that pivot

Deletion of pivotal cases reduces competence

Pivotal

Auxiliary

1

4

2

3Coverage

Set

Competence Categories: Auxiliary

A case is auxiliary if its coverage set is a subset of the coverage of one of its reachable cases auxiliary(c) iff c′

reachable(c) coverage (c) coverage (c′)

Auxiliary cases tend to lie in clusters of cases Deletion of auxiliary cases makes no difference

If one is deleted then a nearby case can be used to solve any target that the deleted auxiliary could solve

PivotalAuxiliary

1

4

2

3

Competence Categories: Spanning

Spanning Cases have coverage sets that link (span) other regions of the problem space

coverage(2) spans coverage of 1 & 3

no more coverage than 1 & 3 but if 3 deleted, 2 is needed

Spanning cases do not directly affect competence But if cases from linked regions deleted the spanning case may be necessary

1

2

3

PivotalSpanningAuxiliary

Competence Categories: Support

Support cases special kind of spanning case exist in groups each support case provides similar

coverage to others in group Deletion of any case in support group

does not reduce competence Deletion of all in group

equivalent to deleting pivot

12 3

A Second Case Competence Model

Competence Group collection of related cases

Two cases belong to the same group if coverage sets overlap i.e., the two cases exhibit shared coverage

Every case belongs to one and only one competence group

Smyth & McKenna

1

3

4

Group 2Group 1

Coverage(1)={1,2}Coverage(2)={1,2,3}Coverage(3)={3}Coverage(4)={4}

2

A Second Case Competence Model

Group Coverage is proportional to size of group

larger groups cover more target problems inversely proportional to density

of cases denser groups cover smaller regions

Case-base Coverage = Group Coverage Predicted competence = case-base coverage

How does real competence relate to predicted competence?

Predicted vs True Competence

Experiments 1000 different cases 300 chosen randomly as unseen problems Other 700 used to build case-bases

True competence % accuracy on unseen

problems compared with predicted

competence

Competence Holes

What is a competence hole? any uncovered region of

the target space What makes a

competence hole interesting? size of the hole relevance to target

problems

Types of Competence Holes

Type 1 - Lost coverage

Insufficient cases within case-base.

Type 2 - No lost coverage

Due to domainconstraints –impossible valuecombinations.

Identifying Interesting Holes

Methodology Competence groups that

are close may merge into a single group

Missing cases are competence rich spanning cases

Search for new spanning cases in the regions between nearby competence groups

Identifying Interesting Holes

Boundary Cases Each pair of groups

has pair of maximally similar cases

gH and hG ( ) for G,H

Each group has n-1 boundary cases

corresponding to the n-1 other groups

Identifying Interesting Holes

For each group search for new spanning cases between it and its nearest neighbour group New case is between

boundary cases

Case Authoring

Building new cases to fill the competence holes in the case-base

Methodology Generate a new case from the feature values of

boundary pair cases For Nominal Features

Choose Most frequent value For Continuous Features

Choose Mean value

Summary

Competence Competence groups Competence holes

Competence based maintenance Case deletion Case authoring

Boundary cases Spanning cases between boundary cases

Increasing the competence of case-bases

Reading

Research papers B. Smyth, M.T. Keane. Remembering To Forget

– A Competence-Preserving Case Deletion Policy for CBR Systems. In Proceedings of IJCAI, pp. 377-382, Canada, 1995. http://www.idi.ntnu.no/emner/it3704/lectures/papers/smyth-keane.pdf

B. Smyth, E. McKenna. Building Compact Competent Case-bases. In proceedings of ICCBR, Munich, Germany. pp. 329-342. Springer Verlag, 1999. http://citeseer.ist.psu.edu/cache/ ... / smyth99building.pdf