Ken Forbus presented “Software Social Organisms: Implications for measuring AI progress.”
Automated Critique of Sketched Mechanisms Jon Wetzel and Ken Forbus Qualitative Reasoning Group...
-
Upload
felicia-haynes -
Category
Documents
-
view
214 -
download
0
Transcript of Automated Critique of Sketched Mechanisms Jon Wetzel and Ken Forbus Qualitative Reasoning Group...
Automated Critique of Sketched Mechanisms
Jon Wetzel and Ken Forbus
Qualitative Reasoning Group
Northwestern University
Outline
• Motivation• Algorithm
– Input: CogSketch
– State Transition Verification• Generate requirements from sketches
• Make predictions using QM
• Verify that predictions reflect requirements
• Evaluation• Related Work• Future Work
Motivation
Engineering Design and Communication Courseat Northwestern University
http://www.segal.northwestern.edu/undergraduate/edc/
Example Sketch of EDC project
Palm-actuated one-handed fingernail clipper
Long-term Goal: Design Buddy
• Given: An explanation consisting of…
• Return:– Is the explanation logically consistent?
– Why or why not?
– Provide feedback that improves users’ communication skills
Sketches Language-like input
+
Outline
• Motivation• Algorithm
– Input: CogSketch
– State Transition Verification• Generate requirements from sketches
• Make predictions using QM
• Verify that predictions reflect requirements
• Evaluation• Related Work• Future Work
Critique of Sketched Mechanisms
Critique is a three-step process
1. Derive a set of state transition requirements using the qualitative representation of a sketch
2. Use qualitative mechanics (QM) to predict outcome of each state• Qualitative mechanics (Wetzel and Forbus, 2008)
based on Nielsen (1988) and Kim (1993)
3. Compare prediction with requirements
Input: CogSketch
• User draws three categories of things– Objects
– Annotations
– Relations
• User labels each using our ontology (based on OpenCyC KB)
• No reliance on recognition, except for arrows
Input: CogSketch
Sketches include knowledge representation
(isa PalmRest RigidObject)
(isa Spring1 Spring-Device)
(forceApplied PalmRest Down) . . .
Input: CogSketch
CogSketch computes spatial relations
(touchesDirectly PalmRest LeverArm)
(connectedTo-Directly Spring1 LeverArm)
(QualitativeVectorBetween (tail ForceArrow1) (head ForceArrow1) Down)
Input: CogSketch
• Multiple subsketches
• One subsketch per mechanical state
Up (open) position
Down (closed) position
Each sketch has a metalayer
• Contains all of the subsketches
• User draws relation arrows indicating cause-effect relationship
Input: CogSketch
Outline
• Motivation• Algorithm
– Input: CogSketch
– State Transition Verification• Generate requirements from sketches
• Make predictions using QM
• Verify that predictions reflect requirements
• Evaluation• Related Work• Future Work
State Transition Verification
• Input: CogSketch sketch with labeled state transitions
• Output: Comparison of requirements and verification1. Sketches depict the required result of previous state
2. Qualitative mechanics predicts state behaviors
3. Verify the predicted behavior makes required results
Step 1/3: Derive Requirements
Spatial relations are computed between subsketches
(relativePositionChange State1 State2 PalmRest ClipperFrame Down)
(q-2D-orientation State1 PalmRest Horizontal)
(q-2D-orientation State2 PalmRest Horizontal)
. . .
Step 1/3: Derive Requirements
Spatial relations become the state transition requirements.
(relativePositionChange State1 State2 PalmRest ClipperFrame Down)
(q-2D-orientation State1 PalmRest Horizontal)
(q-2D-orientation State2 PalmRest Horizontal)
. . .
To go from state1 from state2, PalmRest must move down.
To go from state1 from state2, PalmRest must not rotate.
. . .
RequirementsSpatial Relations
Step 2/3: QM Predicts Behavior
Each requirement becomes a query for the initial state.
To go from state1 from state2, PalmRest must move down.
To go from state1 from state2, PalmRest must not rotate.
. . .
Query: (in State1 (transMotion PalmRest ?dir))How will PalmRest translate?
Query: (in State1 (rotMotion PalmRest ?dir))How will PalmRest rotate?
. . .
Requirements Queries based on requirements
Step 2/3: QM Predicts Behavior
Qualitative Mechanics rules generate prediction• Last year:
– Representation for forces, rigid objects and their surfaces
– Rules propagate forces and motion between surface contacts
• New this year: – Non-rigid body: springs
– Representation for surface textures
– Gears
Based on work by Nielsen (1988) and Kim (1993)
Step 2/3: QM Predicts Behavior
Qualitative Mechanics rules generate prediction
QM
Query: (in State1 (transMotion PalmRest ?dir))How will PalmRest translate?
Query: (in State1 (rotMotion PalmRest ?dir))How will PalmRest rotate?
. . .
(transMotion PalmRest Down))PalmRest will move down.
(rotMotion PalmRest CW))
PalmRest will rotate clockwise.. . .
Result of verificationQueries based on requirements
Step 3/3: Compare Results
Check to see if requirements are verified
(transMotion PalmRest Down))PalmRest will move down.
(rotMotion PalmRest CW))
PalmRest will rotate clockwise.. . .
To go from state1 to state2, PalmRest must move down.
To go from state1 to state2, PalmRest must not rotate.
. . .
Requirements Result of verification
Disagree
Indicates a possible bug in the design or explanation
Agree
Outline
• Motivation• Algorithm
– Input: CogSketch
– State Transition Verification• Generate requirements from sketches
• Make predictions using QM
• Verify that predictions reflect requirements
• Evaluation• Related Work• Future Work
Evaluation
• Selected EDC projects from previous years– Mechanical systems
– Could be easily represented in two dimensions
• Final corpus:16 student projects worked with CogSketch and our QM– 6 were very similar to other projects, so 10 were
sketched
Evaluation
• Success criterion: – Illustrate motions and forces in the design
– Successful state transition verification• All requirements correctly derived from sketch
• QM successfully verifies all requirements
• Result: succeeded on examples representing 15 of the 16 projects
Examples from EDC Projects
Examples from EDC Projects
Examples from EDC Projects
Examples from EDC Projects
Related Work
• SketchIt (Stahovich et al, 1998)
• Recognition: (Alvarado and Davis, 2004; Hammond & Davis, 2005; Kurtoglu and Stahovich, 2002)
• Physics simulation: (Alvarado and Davis, 2001)
Future Work
• Extend and improve QM– Use surface texture representation to implement
friction
– Automatically suggest axes of rotation
– Predict possible outcomes from ambiguous input
• Studies with EDC students this Fall– Add interface for language-like input– Generate feedback which improves communication
skills
Acknowledgements
This research was supported by NSF SLC Grant SBE-0541957, the Spatial Intelligence and Learning Center (SILC)