Selection of Clinical Trials:Knowledge Representation and Acquisition

Committee:

Eugene Fink

Lawrence O. Hall

Dmitry B. Goldgof

Savvas Nikiforou

Automated Matching of Patients to Clinical Trials

Faculty:

Lawrence O. Hall

Dmitry B. Goldgof

Eugene Fink

Part of the project:

Students:Lynn FletcherPrinceton KokkuSavvas NikiforouBhavesh GoswamiTim IvanovskiyRebecca Smith

Expert System

The system analyzes a patient’s data and

determines whether the patient is eligible

for Moffitt clinical trials.

Expert System

• Guides a clinician through related questions

• Identifies appropriate medical tests

• Selects matching clinical trials

• Minimizes pain and cost of selection process

Outline

• Previous work

• Eligibility decisions

• Knowledge base

• Knowledge entry

• Experiments

Previous Work

• Medical expert systems

• Knowledge acquisition

• Medical systems at USF

Medical Expert Systems

• If-then rules:

– Mycin (1972), Puff (1977), Centaur (1977)

• Qualitative reasoning:

– Oncocin (1981), Eon (1995), OncoDoc (1998)

• Bayesian networks:

– Hepar (1990), AIDS2 (1990)

Knowledge Acquisition

• Teiresias (1974): Knowledge for Mycin

• Salt (1985): Elevator-design rules

• Opal (1987): Knowledge for Oncocin

• Protégé (1987, 2000):

General-purpose tools for developing

knowledge acquisition interfaces

Medical Systems at USF

Selection of clinical trials for cancer patients

• Bayesian networks (Theocharous)

• Qualitative reasoning (Fletcher and Hall)

No knowledge acquisition tools

Outline

• Previous work

• Eligibility decisions

• Knowledge base

• Knowledge entry

• Experiments

Example: Eligibility Criteria

• Female, older than 30

• No prior surgery

• Breast cancer, stage II or III

Example: Questions

Sex:

Age:

Female

Male

25

Example: Conclusion

Patient is not eligible

Example: Questions

Sex:

Age: 35

Female

Male

Example: Questions

Cancer stage:

Prior surgery? Yes No Unknown

I

II

III

IV

Example: Conclusion

Patient is eligible

Full Functionality

• Orders and groups the questions

• Considers multiple clinical trials

Old System

• A programmer has to code the questions

New System

• A programmer has to code the questions

• A nurse enters the questions

through a friendly interface

• Problem: Build the interface

Outline

• Previous work

• Eligibility decisions

• Knowledge base

• Knowledge entry

• Experiments

Main Objects

• Questions

• Medical tests

• Eligibility criteria

Types of Questions

• Yes / No / Unknown

• Multiple choice

• Numeric

Examples of Questions

Prior surgery? Yes No Unknown

Cancer stage: I

II

III

IV

Age:

Tests

A medical test answers several questions.

It involves certain pain and cost.

Example Test: Name and Cost

Test name:

Cost: 50.00

Pain: 1

Mammogram

Example Test: Questions

• Yes / No

Question:

Breast cancer?

Example Test: Questions

• Multiple choice

Question:

Cancer stage IIIIIIIV

Options:

Example Test: Questions

• Numeric

Question: Tumor size 0 25 0

Min Max Prec

Eligibility Criteria

• A logical expression that determines eligibility for a specific clinical trial

Example: CriteriaAND

Age > 30

Prior-surgery = NO

OR

Cancer-stage = II

Cancer-stage = III

Outline

• Previous work

• Eligibility decisions

• Knowledge base

• Knowledge entry

• Experiments

Tests and Questions

Adding tests Modifying a test

Adding yes/no questions

Adding multiple choice questions

Adding numeric questions

Deleting questions

Adding Tests

Test name:

Cost: 45.50

Pain: 1

Mammography test

Yes/No M-Choice Numeric Deleting

Adding Modifying

Mammography test

45.5050.00

Modifying a Test

Test name:

Cost:

Pain: 1

Mammogram

Yes/No M-Choice Numeric Deleting

Adding Modifying

Adding Yes/No Questions

Breast cancer?

• Text

Yes/No M-Choice Numeric

Adding Modifying

Deleting

Cancer stage

Adding Multiple Choice Questions

• Text Options

Yes/No M-Choice Numeric

Adding Modifying

IIIIIIIV

Deleting

Adding Numeric Questions

Tumor size

• Text Min Max Prec

250 0

Yes/No M-Choice Numeric

Adding Modifying

Deleting

Deleting Questions

Patient’s age

Cancer stage

Breast cancer?

Tumor size

Yes/No M-Choice Numeric Deleting

Adding Modifying

Cancer stage

Tumor size

Yes/No M-Choice Numeric Delete

Adding Modifying

Deleting Questions

Demo

Eligibility Criteria

Adding eligibility criteria

Selecting tests

Deleting expressions

Editing questions

Defining an expression

Selecting questions

Example: Eligibility Criteria

• Female, older than 30

• Breast cancer, stage II

• Post-menopausal or surgically sterilized

Adding Eligibility Criteria

Adding criteria

Selecting tests

001 Clinical trial A

Trial number Trial name

Deleting expressions

Editing questions

Defining an expression

Selecting questions

Selecting Tests

General questions

Blood test

Mammogram

Biopsy

Urine test

Adding criteria

Selecting tests

Deleting expressions

Editing questions

Defining an expression

Selecting questions

Selecting Questions

I II III IVCancer stage:

Age: From: To:0 15030

Post-menopausal? UnknownNoYes

Surgically sterilized? UnknownNoYes

Adding criteria

Selecting tests

Deleting expressions

Editing questions

Defining an expression

Selecting questions

Prior surgery? UnknownNoYes

Defining an Expression

Cancer-stage = II

Surgically-sterilized = YES

Post-menopausal = YES

Age > 30

Adding criteria

Selecting tests

Deleting expressions

Editing questions

Defining an expression

Selecting questions

Defining an Expression

AND

Adding criteria

Selecting tests

Deleting expressions

Editing questions

Defining an expression

Selecting questions

Cancer-stage = II

Surgically-sterilized = YES

Post-menopausal = YES

Age > 30

Defining an Expression

Adding criteria

Selecting tests

Deleting expressions

Editing questions

Defining an expression

Selecting questions

Surgically-sterilized = YES

Post-menopausal = YES

AND

Age > 30

Cancer-stage = II

Defining an Expression

Adding criteria

Selecting tests

Deleting expressions

Editing questions

Defining an expression

Selecting questions

Surgically-sterilized = YES

AND

Age > 30

OR

Post-menopausal = YES

Cancer-stage = II

Defining an Expression

Adding criteria

Selecting tests

Deleting expressions

Editing questions

Defining an expression

Selecting questions

AND

Age > 30

OR

Post-menopausal = YES

Cancer-stage = II

Surgically-sterilized = YES

Demo

Outline

• Previous work

• Eligibility decisions

• Knowledge base

• Knowledge entry

• Experiments

Experiments

Performance of seven novice users

• Entering tests and questions

• Entering eligibility criteria

0

20

40

60

80

100

0 1 2 3 4

number of a test set

time

per

ques

tion

(sec

)Entering Tests and Questions

Learning curve

Entering Eligibility Criteria

Learning curve

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8 9 10 11

number of a clinical trial

time

per

ques

tion

(sec

)

Entering Eligibility Criteria

0

200

400

600

800

1000

1200

1400

1600

0 5 10 15 20 25 30 35

number of questions

entr

y tim

e (s

ec)

Summary

• Learning time: 1 hour

• Adding a test: 2 to 10 minutes

• Building a knowledge base for Moffitt

breast-cancer trials: 8 to 10 hours

• Adding eligibility criteria: 30 to 60 minutes

Main Results

• Formal model of selection criteria

• Representation of related knowledge

• Friendly interface for knowledge entry

Future Work

• Probabilities of different answers

• Logical connections among questions

• Detection of identical and related questions