© 2006 Kotovsky and Cagan Sources of Insights in Engineering Design Kenneth Kotovsky, Dept. of...

© 2006 Kotovsky and Cagan

Sources of Insights in Engineering Design

Kenneth Kotovsky, Dept. of Psychology

and

Jonathan Cagan, Dept. of Mechanical Engineering Carnegie Mellon University


Design Insights

• How do creative insights occur during the design process?

• Can their frequency and quality be enhanced?

• What are the underlying cognitive mechanisms?


Overview

• A conceptual framework or model for design problem-solving

• Brief introduction of that framework

• Examples of some of the work that led to it

• Presentation of some current work within it


A Multi-plane Model of Design

• Search is viewed as occurring within a representation depicted as a plane

• Multiple planes equiv. to different representations • Search for new representations/planes activated

by impasses often leading to insights• Search can involve both conscious and non-

conscious processes often operating in parallel or resulting in parallel search within multiple planes


A Multi-plane Model of Design

*

impasse

solution

Mov

emen

t to

new

rep

rese

ntat

ions


Prior work has demonstrated two distinct phases of problem solving; exploratory and insight-like final path

5 4 3 2 1

Ball Configuration

31

30

29

28

27

26

25

24

23

22

21

20

19

18

17

16

15

14

13

12

11

10

9

8

7

6

5

4

3

2

1

0

StateProblem Space

31 Move Start

21 Move Start

Goal

Non-Conscious Insight Attainment

Isomorphs of Chinese Ring Puzzle


Empirical Results

Kotovsky, K. & Simon, H.A. (1990) Why are Some Problems Really Hard: Explorations in the Problem Space of Difficulty. Cognitive Psychology, 22, 143-183


Final Path: Non-Conscious Insight

• Now I want to ask you about the puzzle you just solved, how it worked, what you did.

• No idea.• No idea?• No idea. It was very painful, (laughs)• You did get it, right?• Yeah, but it was basically luck... that I got it.• You had no idea what you were doing?• Not really.• Suppose somebody else was going to do the puzzle who had never seen it

before and you had to give them some hints, tell them how to solve it.• Well, let's see ... I don't know what to say ... but, I guess, (garbled word) the

puzzle ...The good part was that there usually wasn't any more than like, one or two choices. I think there was one choice. Was there any more than one choice? I don't know. But I had (garbled word). Which is why I kept ending up back where I started from which was frustrating. I would tell them, I would tell them, good luck. That's all.

Reber, P. & Kotovsky, K. ((1997) Implicit Learning in Problem Solving: The Role of Working Memory Capacity. JEP, General. 126, 178-203


Computational Model of Two-Stage Problem Solving

• Simulated annealing algorithm

• Works within the problem space with minimal assumptions about that space

• Generates objective function through experience

• Improves over multiple runs

Cagan, J. & Kotovsky, K. (1997) Simulated Annealing and he Generation of the Objective Function: A Model f Learning During Problem Solving. Computational Intelligence 13:4 534-581


Computational Model of Two-Stage Problem Solving: Ball & Boxes

B&B - Misleading

move no.

problem state05

1015202530

050

100 150 200 250

B&B - 1st Ball Out

move no.

problem state0

10

20

30

40

050

100 150 200 250

Cagan, J. & Kotovsky, K. (1997) Simulated Annealing and he Generation of the Objective Function: A Model of Learning During Problem Solving. Computational Intelligence 13:4 534-581


Summary: Non-Conscious Insight Attainment

• Dichotomous: Exploratory & Final Path

• Looks insightful

• Non-conscious insights– Also supported by work on

• Subliminally flashed hints (with Julia Lin)• Delayed realization of having reached an

insight/solved a problem (with Michelle Fahey)


Multi-plane Model of Design

Some candidates for “Plane-jumping” or insightful representation change include:– Reaching an impasse

– Noticing relevant information

– Incubation

– Relaxing the most restrictive constraint.

– Using and tracking the use of analogy to generate other search spaces or planes


Incubation Impasses & Open Goals

(with Jarrod Moss)

• Open goals related to prepared mind view of insight

• Affect behavior on a range of tasks, starting with Zeigarnik Effect (1927) & long line of work since (ex. Patalino & Seifert (1994, 1997)

• Attempt to demonstrate open goal effect and explore its functioning in impasses

Moss, J. (2006)The Role of Open Goals in Acquiring Problem Relevant Information. Ph.D. Dissertation, Carnegie Mellon University


Study 1a• 39 Participants• 30 seconds to work on Remote Associate

Test (RAT) problems (Mednick, 1962)– Short insight-like problems (fox peep man) hole– Fixation on distractors (Smith & Blankenship, 1991)

• Answers for unsolved problems presented in lexical decision task (hint)

RATProblems

(20)

RATRepeated

(20)

Lexicaldecision task

(50)


Lexical Decision

550.00

560.00

570.00

580.00

590.00

600.00

610.00

620.00

630.00

640.00

Solved Unsolved

RAT Result

RTControl

RAT


Unsolved Problems Solved on Second Attempt

0.00

0.05

0.10

0.15

0.20

0.25

0.30

Previously Unsolved Problems

Proportion Solved

No Hint

Hint


Study 1b

• 30 Participants

• Method similar to Study 1a– Hints after unsolved problems

– Hints before new problems

RAT Problems RAT Problems

Repeated+ New

Lexical decision task


Effect of hint for new and unsolved problems

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

New Previously Unsolved

Type of problem

Proportion solved

No Hint

Hint

22% relative improvement

50% relative improvement


Effect of hint on solution time (solved problems)

0.00

5.00

10.00

15.00

20.00

25.00

New Previously Unsolved

Type of problem

Solution time in seconds

No Hint

Hint


Results

• Hint more effective with open goals– 50% relative increase vs. 22%

– Improvement in solution time

– Majority of participants unaware of any relationship between the two tasks (> 90%)

– Also did study showing sizable no. of open goals could be maintained


Study 2 Impasses: Protocol Study • Hint vs. no-hint• Concurrent Verbal protocols

– 30 participants

– Assessment of candidate generation & fixation

– Measured Impasses (10 sec. or throwing up hands)

• Varying amount of time spent on a problem– 15, 30, 60 seconds 1st attempt, 30 seconds 2nd attempt

RATProblems

(30)

RATRepeated

(30)

LexicalDecision task

(50)


Fixation on Distractors

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Unsolved Improved

Result of Second Attempt

Proportion of Candidates that are Repeats

No Hint

Hint


Impasses

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Impasse No Impasse

Type of Impasse

Proportion Correct

No hint

Hint


Impasses: Exploration and Fixation

15 s, Hint

30 s, Hint

60 s, Hint

No Hint

Interactions:

(1) Hint xTime Limit

(2) Hint x Candidates

Time limit for second attempt

always 30 s


A Conceptualization of Impasses

Time

Exploration

Fixation


Model• Probability of retrieval

– Activation of word– Activation of distractors leading to fixation

• Fixation– Retrieving a word increases base level activation– Multiple retrievals result in greater increase– Interference as more words are retrieved

• Implemented in ACT-R• Shows that activation and fixation alone can’t

account for results, need open goal mechanism


Results & Conclusions• Hints are effective in presence of open goals that

act by sensitizing us to information from the environment– Less effective with more fixation– Without hint, fixation continues into 2nd solution attempt

• We can assimilate hints without awareness• Redefining impasse as a more continuous process

due to interplay of– Fixation– Exploration

• We’ve seen some of the same impasse related fixation on engineering design problems

• Compatible with other work showing non-conscious problem solving leading to multi-plane search


Future Directions• Map findings onto the design of design

activities so as to optimize: – Timing via tracking exploration & fixation

– Exposure to potentially useful information to increase range and frequency of creative insights

• This research thrust is itself a multi-plane approach as we try to both enhance our understanding of cognitive mechanisms and apply them to engineering design activities and computational implementations.


Bibliography• Mednick, S. A. (1962). The asssociative basis of the creative

process. Psychological Review, 69, 220-232• Patalano, A. L., & Seifert, C. M. (1994) Memory for impasses

during problem solving. Memory & Cognition 22(2), 234-242• Patalano, A. L., & Seifert, C. M. (1997) Opportunistic planning:

Being reminded of pending goals. Cognitive Psychology 34(1), 1-36.

• Smith S. M., & Blankenship, S. E. (1991) Incubation and the persistence of fixation in problem solving. American Journal of Psychology, 104(1), 61-87.

• Zeigarnik, B. (1938). On finished and unfinished tasks. In W. D. Ellis (Ed.), A source book of gestalt psychology (pp 300-314). New York: Harcourt-Brace. (Reprinted & condensed from Psychologische Forschung, 9, 1-85, by B. Zeigarnik, 1927.


Acknowledgements• Jarrod Moss • Julia Lin• Michelle Fahey• Paul Reber• Matt Campbell• Herbert Simon• J.R. Hayes

• AFOSR under grant # FA9620-04-1-0201

The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of AFOSR or the U.S. Government.

© 2006 Kotovsky and Cagan Sources of Insights in Engineering Design Kenneth Kotovsky, Dept. of...

Documents

Transcript of © 2006 Kotovsky and Cagan Sources of Insights in Engineering Design Kenneth Kotovsky, Dept. of...