Dark knowledge in Qualitative Reasoning: A call to Arms Ken Forbus & Dedre Gentner Northwestern...
-
date post
18-Dec-2015 -
Category
Documents
-
view
215 -
download
2
Transcript of Dark knowledge in Qualitative Reasoning: A call to Arms Ken Forbus & Dedre Gentner Northwestern...
Dark knowledge in Qualitative Reasoning: A call to Arms
Ken Forbus & Dedre Gentner
Northwestern University
Example of the Problem
• Currently CO2 entering the atmosphere exceeds rate of its removal– Therefore it accumulates
– Stopping emissions growth (i.e., freezing emissions at their current level) would not change this
• Steerman (2009) found that 84% of participants drew response curves to CO2 in atmosphere that showed atmospheric CO2 stabilizing. – 3/5ths had degrees in science, engineering, or
mathematics (rest: economics)
– 30% had advanced degrees, 70% of which were in science, engineering, or math
Mental Models Matter
• Climate change– “Wait and see” attitude more prevalent in countries
catching up economically
– Save the planet?
• Financial meltdown– People resist the idea that markets do fall
• Evidence of failures in education– Policy influences economics fog produced by special
interests
– Education as an adversarial game
Hypothesis: QR can help
• Formalisms for causal qualitative models• Thinking tools at natural level of abstraction• Some attempts have been made
– Betty’s brain
– Vmodel
– GARP models of Brazilian ecosystem phenomena.
• Not much headway yet. Why?– Not enough effort expended yet?
– Not enough time to diffuse and have impact?
– Claim: We need to take a more psychologically realistic look at how mental models work
Overview
• How mental models work (we think)– Experience = the dark knowledge that binds our
conceptual universe together
• How to improve human qualitative reasoning– Analogy as mechanism for integrating knowledge
– Example: Bathtubs and climate change
– Example: Bathtubs and credit card debt
• What to do?– Understand better when and how experience is used
– Create tools that can exploit experiential knowledge as well as abstract domain theories
Standard QR model
1st principlesDomain Theory
SituationDescription
Model Builder QualitativeSimulator
ScenarioModel
Predictions ofpossible behaviors
Alternative: Experience-based modeling
• People store and use episodic knowledge– True in all parts of human cognition, why would QR be
different?
– Provides robustness, heuristics for quick decisions, reality checks
• Q: How much folic acid does a pregnant woman need?
• A: Six tons (TREC system)
• Analogy provides means of directly reasoning with episodic knowledge– Similarity-based retrieval works well for within-
domain analogs
– No need to make knowledge into rules before using it
Culture as a Source of Experience
• Many things we understand we’ve never directly experienced– Revolutionary War, molecular structure, plate
tectonics, evolution
• Vygotsky: Much of our knowledge is learned by interactions with other people– Apprenticeship
– Reading, other media
• Implication: Our systems need fluid interaction with human teachers, learners & collaborators
Experience = Dark Knowledge in QR
• In physics,– Dark matter outweighs the matter we can see
– Its gravitational force is what holds the universe together
• In QR,– We’ve focused on abstract, 1st principles knowledge
– Experiential knowledge in human cognition far outweighs the abstract domain theories that we tend to focus on
– Its use in heuristic and analogical reasoning is what holds our conceptual universe together
Experience-based QR (one version)
Experience
1st principles domain theory
Current Situation
Standard QR
Analogical Retrieval
Predictions
Predictions
What Happens
Analogical Generalization
Learning mental models
• Experiences → Protohistories → Causal models → Naïve Physics → Expert Models– Forbus & Gentner, 1986; 1997
– Recall Scott Friedman’s talk
NL
Temporally encodedCycL exemplars
def-EH: participants: … conditions: …
consequences: …
def-EH: participants: … conditions: …
consequences: …
New Scenarios
NL
Reasoning
List of target concepts
{pushing, moving, blocking}
SEQL Generalizations
EncapsulatedHistories
Comic-strip stimuli
Protohistories
Causal models
Analogies
http://throbgoblins.blogspot.com/2009/01/boffin-bathtub-again.html
Analogy as knowledge integration
• Students often “check their intuition at the door”• Comparing their analyses with everyday
experience could improve accuracy• But only if they understand the everyday world
– E.g., Gentner & Gentner (1983), Some participants in water-electricity analogy study didn’t understand basic facts about water pressure
Stocks & Flows• Sweeney & Steerman, 2000: Bathtub Dynamics
The bathtub initially contains 100 liters.
Draw the behavior of the quantity of water on the lower graph
Stocks & Flows• Sweeney & Steerman, 2000: Bathtub Dynamics
A correct student answer
Typical Incorrect Answers
Techniques for using analogy effectively
• Make sure the base domain is well understood• Work through the correspondences explicitly and
carefully– Richland et al 2009, in studying use of math analogies
across instructors in US, Japan, and China
• Having learners compare cases can double transfer effectiveness, including own memory retrieval– Gentner et al 2003, in preparation
Bathtubs, revisited
Bathtubs and Credit Cards
Bathtub Credit Card
Faucet Setting Monthly charges
Drain setting Monthly payment
Level of water Amount of debt
??? Interest rate
Projecting backwards to understand differences
Mr. Will on ice
• George Will, Washington Post, 2/15/09:“As global levels of sea ice declined last year, many experts said this was evidence of man-made global warming. Since September, however, the increase in sea ice has been the fastest change, either up or down, since 1979, when satellite record-keeping began.. According to [authority], global sea ice levels now equal those of 1979.”
Mr. Will on ice
• George Will, Washington Post, 2/15/09:“As global levels of sea ice declined last year, many experts said this was evidence of man-made global warming. Since September, however, the increase in sea ice has been the fastest change, either up or down, since 1979, when satellite record-keeping began.. According to [authority], global sea ice levels now equal those of 1979.”
“No single year marks a trend or holds evidence of long-term
climate change.”-- Andrew Revkin, NYT dot-earth
blog
QR can be subtle
• Jennifer Francis, Rutgers:“At the end of summer each year, the sea ice refreezes and continues to do so until late spring. Thin ice and open water generate new ice faster than thick ice, as heat from the ocean below is able to escape more easily into the atmosphere. In the autumn of 2007 and 2008, the rate of ice production was very large because there was so much open water and thin ice – The rapid growth is completely expected.
What is to be done?
• Better understanding of the nature of experiential knowledge in QR
• Build better tools to support reasoning and learning– QR tools
– Analogy tools that combine QR and episodic knowledge.
EA NLU provides psychologically
plausible language outputs
Building Experience-based Systems
COMLEXlexicon
ResearchCycKB Contents
Allen Parser
Query-based
AbductiveSemantic
Interpreter
DRT-basedPacked
SemanticsBuilder
Formal representation
of story
Then child child13 is playing with
the truck truck13.
CogSketch provides
psychologically plausible visual
and spatial processing
Better tools
• QR systems need to exploit concrete, experiential knowledge more– Garp3, Gizmo, Envision – all require abstract domain
theories, & don’t support concrete knowledge.
– Betty’s Brain, Vmodel – concrete knowledge only, no logical quantification.
– QCM bet: Easier for everyone to encode concrete knowledge first
• Analogy- based tools– Help people work through analogies
– Suggest potential analogs
Summary
• Two of our current crises are caused in part by mental model deficiencies– Even technically trained people do not understand
accumulation of CO2 in the atmosphere
– Our financial crises rely on a variety of faulty models
• Experiential knowledge is the “dark matter” of QR– It far outweighs abstract domain theories, and provides the
forces that hold our conceptual universes together
• QR research can help– Need to better understand how experiential knowledge
works
– Need to build better tools, that incorporate experiential knowledge and support analogical reasoning
Questions, suggestions?