PROBLEM SOLVING AND METACOGNITION · Problem(solving(and(metacognition(!! (! problem.!
Epistemological Framework for Reuse Problem Solving Method Generic Task Multi-Functional Domain...
-
date post
21-Dec-2015 -
Category
Documents
-
view
223 -
download
0
Transcript of Epistemological Framework for Reuse Problem Solving Method Generic Task Multi-Functional Domain...
Epistemological Framework for Reuse
Problem SolvingMethod
Generic Task
Multi-Functional Domain
MappingKnowledge
Application-specificProblem-Solving Knowledge
Application Configuration
Domain Ontology
Task Ontology Method Ontology
Mapping Ontology
Application Model
A Library of Components for Classification Problem Solving
Main Goals
• To carry out a knowledge-level analysis of classification
• To develop a practical resource to support the development of classification applications
• To provide a concrete set of components to act as a test case for IBROW brokering system and IRS
Classification
Classification can be seen as the problem of finding the solution (class), which best explains a set of known facts (observables), according to some criterion
Observables
Candidate Sols.Classification Solution
Solution Criterion
Example
Observables
Candidate Sols.
Criterion
Classification Solution
{background=green; area=china...}
Complete-coverage-criterion(every observable has to be explained)
{chinese-granny, dutch-granny, etc..}
{chinese-granny}
Observables
Observables = set_of (Observable);Observable = {feature, value}.
Well defined Observables (obs):
({f1, v1} obs {f1, v2} obs) -> v1 = v2
({f1, v1} obs) -> legal_feature_value (f1, v1 )
Solutions
Solution = set_of (Feature_Spec);Feature_Spec = {Feature, Feature_value_spec}Feature_value_spec = Unary_Relation
Well defined Solution (sol):{f1, s1} sol holds (s1, v1 ) ->
legal_feature_value (f1, v1 )
Matching
Observable={f1, v1} matches Solution=sol iff:
{f1, c} sol holds (c, v1 )
Matching Sets of Obs to a Solution
Sol: {{fsol1, c1}...{fsolm, cm}}; Obs: {{fob1, v1}...{fobn, vn}}
Four possible cases: {fj, cj} sol {fj, vj} obs holds (cj, vj)
-> Explained (fj)
{fj, cj} sol {fj, vj} obs not holds (cj, vj) -> Inconsistent(fj)
{fj, vj} obs {fj, cj} sol -> Unexplained (fj)
{fj, vj} obs {fj, cj} sol -> Missing (fj)
Default Match Criterion
Match Score:Vector: <I, E, U, M>
Match Comparison RelationS1 = (i1, e1, u1, m1); S2 = (i2, e2, u2, m2)
S1 better_score than S2 iff:
(i1 < i2)
(i2 = i1 e2 < e1) (i2 = i1 e2 = e1 u1 < u2) (i2 = i1 e2 = e1 u2 = u1 m1 < m2)
Other Match Criteria
• SUBSET-BASED-MATCH-CRITERION– Uses subset instead of < to determine best
match
• ABSTRACTION-AWARE-MATCH-CRITERION – Matching mechanism able to handle both
observables and abstracted data
• EXPLANATION-CENTRED-MATCH-CRITERION– Focuses on explanation power
• EQUAL-RATING-MATCH-CRITERION– Very stupid one, which gives every solution the
same score
Possible Solution Criteria
• Positive Coverage– Some feature is explained and none is
inconsistent
• Complete Coverage– All features are explained and none is
inconsistent
• No missing features – No inconsistent features and no missing
features
Hierarchy of Criteria
Solution Criterion
Match Criterion
Match Score Comparison Rel
Macro Score MechanismFeature Score Mechanism
Match Score Mechanism
Observables
(def-class observables (set) ?obs "This is simply a set of observables. An important constraint is that there cannot be two values for the same
feature in a set of observables" :iff-def (every ?obs observable) :constraint (not (exists (?ob1 ?ob2) (and (member ?ob1 ?obs) (member ?ob2 ?obs) (has-observable-feature ?ob1 ?f) (has-observable-feature ?ob2 ?f) (has-observable-value ?ob1 ?v1) (has-observable-value ?ob2 ?v2) (not (= ?v1 ?v2))))))
Solutions
(def-class solution () ?x "A solution is a set of feature definitions" :iff-def (every ?x feature-definition))
(def-class feature-definition () ?x ((has-feature-name :type feature) (has-feature-value-spec :type unary-relation)) :constraint (=> (and (has-feature-name ?x ?f) (has-feature-value-spec ?x ?spec)) (=> (holds ?spec ?v) (legal-feature-value ?f ?v))))
Solution Criterion
(def-class solution-admissibility-criterion () ?c ((applies-to-match-score-type :type match-score-type) (has-solution-admissibility-relation :type unary-relation)) :constraint (=> (and (solution-admissibility-criterion ?c) (has-solution-admissibility-relation ?c ?r) (domain ?r ?d)) (subclass-of ?d match-score)))
Monotonicity of Admissibile Solutions
(def-axiom admissibility-is-monotonic "This axiom states that the admissibility criterion is monotonic. That is, if a
solution, ?sol, is admissible, then any solution which is better than ?sol will also be admissible"
(forall (?sol1 ?sol2 ?obs ?criterion) (=> (and (admissible-solution ?sol1 (apply-match-criterion
?criterion ?obs ?sol1) ?criterion)
(better-match-than ?sol2 ?sol1 ?obs ?criterion)) (admissible-solution ?sol2 (apply-match-criterion
?criterion ?obs ?sol2) ?criterion))))
Complete Coverage
(def-instance complete-coverage-admissibility-criterion solution-admissibility-criterion ((applies-to-match-score-type default-match-score) (has-solution-admissibility-relation complete-coverage-admissibility-relation)))
(def-relation complete-coverage-admissibility-relation (?score) "a solution should be consistent and explain all features" :constraint (default-match-score ?score) :iff-def (and (= (length (first ?score)) 0) ;;no inconsistency (= (length (third ?score)) 0))) ;;no unexplained
Classification Task Ontology
• 70 Definitions• Provides both a theory of classification and a
vocabulary to describe classification problems• Ontology is separated from task specifications
Generic Classification Task
• Input roles– Candidate Solutions, Match Criterion, Solution
Criterion, Observables
• Precondition– Both observables and candidate solutions have
to be provided
• Goal– To find a solution from the candidate solutions
which is admissible with respect to the given observables, solution criterion and match criterion
(def-class classification-task (goal-specification-task) ?task "Classification is defined here as finding one or more admissible solutions out of a predefined solution space, which explain the features of a given set of observables, in accordance with a given match criterion and solution admissibility criterion. Because different variants of the goal can be formulated, the goal of the task is given here only as a default" ((has-input-role :value has-candidate-solutions :value has-observables :value has-match-criterion :value has-solution-admissibility-criterion) (has-output-role :value has-solutions) (has-candidate-solutions :type solution-space) (has-observables :type observables) (has-match-criterion :type match-criterion :default-value default-match-criterion) (has-solution-admissibility-criterion :type solution-admissibility-criterion :default-value default-solution-admissibility-criterion) (has-solutions :type solution-space) ……………….))
(has-precondition :value (kappa (?task) (exists (?x ?y) (and (member ?x (role-value ?task 'has-observables)) (member ?y (role-value
?task 'has-candidate-solutions))))))(has-goal-expression :default-value (kappa (?task ?sols) (forall ?sol (=> (member ?sol (role-value ?task 'has-solutions)) (admissible-solution ?sol (apply-match-criterion (role-value ?task 'has-match-criterion) (role-value ?task 'has-observables) ?sol) (role-value ?task 'has-solution-admissibility-criterion))))))
Specific Classification Tasks
• Single-Solution Classification Task– Single-solution assumption
• Optimal Classification Tasks– Goal requires optimality
Problem Solving Library
• Based on heuristic classification model• Includes both data-directed and solution-
directed methods• Supported by a method ontology
Method Ontology: Main Concepts
• Abstractors– Mechanism for performing abstraction on
observables– Abstractor: Obs* -> Obs
• Refiners– Mechanism for specialising a solution– Refiner: Sol -> Sol*
• Candidate Exclusion Criterion– A criterion which is used to decide when a
search path is a dead-end– Default criterion rules out inconsistent
solutions
Monotonicity of Exclusion Criterion
(def-axiom exclusion-is-monotonic (forall (?sol1 ?sol2 ?obs ?criterion) (=> (and (ruled-out-solution ?sol1 (the-match-score ?sol1) ?criterion) (not (better-match-than ?sol2 ?sol1 ?obs ?criterion))) (ruled-out-solution ?sol2 (the-match-score ?sol2)?criterion))))
Axiom of Congruence
(def-axiom CONGRUENT-ADMISSIBILITY-AND-EXCLUSION-CRITERIA (forall (?sol ?task) (=> (member ?sol (the-solution-space ?task)) (not (and (admissible-solution ?sol (the-match-score ?sol) (role-value ?task 'has-solution-admissibility-criterion)) (ruled-out-solution ?sol (the-match-score ?sol)
(role-value ?psm
'has-solution-exclusion-criterion)))))))
Three Heuristic Classification PSMs
• Two Data-directed– Admissible Solution Classifier
• Finds one admissible solution according to the given criteria• Uses backtracking hill climbing
– Optimal Classifier• Performs complete search looking for optimal solution• Uses best-first strategy• Uses candidate exclusion criterion to prune search space
• One Solution-directed– Goes down the solution hierarchy, acquiring
observables as needed– Ask for observables with max discrimination power
Four Assumptions in Main PSMs
• No cycles in abstraction hierarchy• No cycles in refinement hierarchy• At least one class in the solution space is an
admissible solution• The solution refinement hierarchy is consistent
with the candidate exclusion criterion. That is if sol is ruled out, all refinements of sol can also be ruled out
Task-Method Hierarchy
abstraction
heuristic-classification-psm
classification
rank-solutions refinement
basic-heuristic-matchselect-abstractor one-step-abstraction collect-refiners apply-refiners
abstraction-psm refinement-psmrank-solutions-psm
Example
• Apple Domain– Originally developed in Amsterdam
• Solutions = Apple Types = {granny, noble, delicious...}
• Hierarchy of Apple Types• Features = {bkg-colour, fg-colour, rusty....}• Pretty trivial really!
Classification TaskOntology
Heuristic ClassificationOntology
Apple Heuristic ClassificationApplication
Classification TaskSpecification
Classification-to-Class-RepresentationMapping Ontology
AppleDomain Model
Heuristic ClassificationPSMs
Mapping Solutions and Obs to Apples
(def-relation-mapping solution :up ((solution ?x) if (or (= ?x apple) (subclass-of ?x apple))))
(def-relation-mapping observable :up ((observable ?x) if (or (== ?X (?f ?v ?obs)) (== ?x (?f ?v)))))
More Relation Mappings
(def-relation-mapping has-observable-feature :up ((has-observable-feature ?x ?f) if (or (== ?X (?f ?v ?obs)) (== ?x (?f ?v)))))
(def-relation-mapping has-observable-value :up ((has-observable-value ?x ?v) if (or (== ?X (?f ?v ?obs)) (== ?x (?f ?v)))))
(def-relation-mapping directly-abstracts-from :up ((directly-abstracts-from ?ob ?obs) if (== ?ob (?f ?v ?obs))))
Sample Abstractor
(def-instance sugar-abstractor abstractor ((has-body '(lambda (?obs) (in-environment ((?v . (observables-feature-value ?obs 'sugar))) (cond ((>= ?v 70) (list-of 'sweet-level 'high (list-of (list-of 'sugar ?v)))) ((and (< ?v 70) (> ?v 40)) (list-of 'sweet-level 'medium (list-of (list-of 'sugar ?v)))) ((<= ?v 40) (list-of 'sweet-level 'low (list-of (list-of 'sugar ?v)))))))) (applicability-condition (kappa (?obs) (member 'sugar (all-features-in-observables ?obs))))))
Generic (reusable) Refiner
(def-instance refinement-through-subclass-of-links refiner "If the solution space is specified by means of classes arranged in a
subclass-of hierarchy, then this is a good refiner to use" ((has-body '(lambda (?sol) (setofall ?sub (direct-subclass-of ?sub ?sol)))) (applicability-condition (kappa (?sol) (and (class ?sol) (exists ?sub (direct-subclass-of ?sub ?sol)))))))
Evaluation/Results
• Library provides an analytical tool to understand classification problem solving
• Library tested on a number of domains– ECAI Paper classification– Paleontology– Medical Diagnosis
• No changes the organization of library needed– Only adding new components, e.g. new match
criteria
• Used by van Harmelen and Ten Teije to study automatic PSM adaptation