BioHealth Informatics Group QA of Ontologies OWL Tutorial December 6 th 2005.
-
Upload
montana-edgin -
Category
Documents
-
view
215 -
download
1
Transcript of BioHealth Informatics Group QA of Ontologies OWL Tutorial December 6 th 2005.
BioHealthInformaticsGroup
QA of OntologiesQA of Ontologies
OWL Tutorial
December 6th 2005
BioHealthInformaticsGroup
The GIGO ProblemThe GIGO Problem
Logic may be necessary
But it is not sufficient
Can not validate all possibleinferences in advance
Instead must prove:Reasoner is soundAxioms are correctInduction does the rest
BioHealthInformaticsGroup
Barriers to Ontology QABarriers to Ontology QA
►Absolute Measures►No ‘Gold Standard’
►Mutual cross validation only as good as the parts
►Manual checking error-prone
►and can’t measure HOW error prone because of (1)
►Comparitive Measures►Better than worse does not imply good
►Relative Measures►Provides unequivocal evidence of improvement
►But not of proximity to goal
►Falling error detection rate does not imply none exist
BioHealthInformaticsGroup
Types of Axiom QualityTypes of Axiom Quality
Philosophical Rigour
Ontological commitment
Content correctness
Fitness for purpose
BioHealthInformaticsGroup
Philosophical RigourPhilosophical Rigour
►2500 years of research
►Theories of time, mereonomy, containment►Often FOL, so not computable
►Similar upper level ontologies ►DOLCE, BFO
►But not 100% agreement: Realist vs Cognitivist
BioHealthInformaticsGroup
Ontological commitmentOntological commitment
►Formally specified semantic equivalence►Logical transformation to canonical form
►Semantically equivalent but no logical transform
► ‘Fixation of femur by means of inserting pins’
► ‘Insertion of pins to fixate the femur’
►Metamodel rules/commitments►Arbitrary choice of preferred form
►Conventions to be applied throughout ontology
►And by all applications that use it
BioHealthInformaticsGroup
Content correctnessContent correctness
► Metadata Provenance, lexical annotations etc
► Truth► ‘Structure of labial vein’ is-a ‘Superficial vein of face’
► Completeness – ambiguity and omission► Thymus secretes Thymosin; Thymosin is-a Hormone
…but omits Thymus is-a endocrine gland
► Conciseness► Redundant inclusion of inferrable axioms
► Consistency – contradictory, duplicated, circular► endocrine surgery vs endocrine surgeons
► Traumatic unilateral amputationUnilateral traumatic amputation
BioHealthInformaticsGroup
Fitness for PurposeFitness for Purpose
► Best theories no guarantee of usability or utility
► Lab experiences no predictor of field behaviour
► All for nothing, if user can’t use it► Interrater variability
BioHealthInformaticsGroup
Fitness for Purpose:Fitness for Purpose: Inter-rater variabilityInter-rater variability
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Headcloth
Cloth
Scarf
Model Person
Woman
Adults
Standing
Background
Brown
Blue
Chemise
Dress
Tunics
Clothes
Suitcase
Luggage
Attache case
Brass Instrument
French Horn
Horn
Tuba
BioHealthInformaticsGroup
Fitness for Purpose:Fitness for Purpose: Inter-rater variabilityInter-rater variability
► Miscoding ► Code meaning is inappropriate to thing being described
► Instrument definitely not a french horn
► Missed coding► Not coding something that could be coded
► No code for the table/platform
► Overcoding► Code meaning is more detailed than justified
► Can the gender really be determined?
► Undercoding► Code meaning is less detailed than justified
► Brass Instument vs Tuba
BioHealthInformaticsGroup
Fitness for PurposeFitness for Purpose
What ontological properties..What ontological properties..
► Increase usablility and utility?
► Are a prerequisite for them?
► Decrease usability or utility?
BioHealthInformaticsGroup
Ontology QAOntology QAHow much quality do we need?How much quality do we need?
Perfection is unattainable
Trade-offs between quality and…Performance
Cost
Maintainability
Usability
Acceptance
Utility
BioHealthInformaticsGroup
Need for CQI, not final QANeed for CQI, not final QAAssure the Assure the pprocess rocess notnot the product the product
Use/Test cases & exemplars
REVIEWExplore
consequences
APPLYQI
Algorithm
DEVISEQI
Algorithm
ASSESSIdentifyproblems