The preference effect in design concept evaluation

Corresponding author:

Jan B. Nikanderjan.nikander@

helsinki.fi

Please cite this article in pre

Studies (2014), http://dx.doi.

e effect in design concept
The preferencevaluation
Jan B. Nikander and Lassi A. Liikkanen, Helsinki Institute for Information

Technology HIIT, Aalto University, PO Box 15600, FI-00076 Aalto, Finland

Miko Laakso, Aalto University Design Factory, Aalto University, PO Box

17700, FI-00076 Aalto, Finland

Concept selection is among the most important activities in new product

development, as the consequences of a poor choice may be disastrous at worst.

These decisions made in the early phases of design processes are, however,

poorly understood from a psychological point of view. This study set out to

extend the tradition of experimental decision-making research into the field of

design. We investigated whether designers systematically prefer their own ideas

in concept evaluation. An experiment with eighteen professional designers was

carried out to test the hypothesis. The findings show a systematic preference of

self-generated concepts in evaluation tasks. We discuss the implications of this

preference effect on design practice and the need for further studies on the topic.

� 2014 Elsevier Ltd. All rights reserved.

Keywords: design behaviour, decision making, conceptual design, ownership

effect in design, protocol analysis

Bynature, human beings have a tendency towards biased decision mak-

ing and apparent non-normative behaviour (see, e.g., Kahneman &

Tversky, 1973, 1979; Stanovich & West, 1998, 2000). We tend to

misinterpret statistical data, make decisions according to insufficient evidence,

interpret information in a way that confirms our preconceptions and become

fixated on information retrieved from memory (Hammond, Keeney, & Raiffa,

1998). It is well accepted that these effects are universal and that they pene-

trate every field of life. The problems of rational and normative decision mak-

ing are evident when dealing with the design and development of new

products. The problems designers deal with are commonly described as ill-

defined or ‘wicked’ problems (Rittel & Webber, 1984), which typically have

no definitely correct solutions that can be identified beforehand and the qual-

ity of the solutions can often be assessed only in retrospect. Consequently,

rational models of problem solving are considered unfit for design (Sch€on,

1983)with designers being susceptible to a number of psychological pitfalls

(Kihlander, 2011).

An extensive body of research on decision making in design exists (see, e.g.,

Ball & Ormerod, 1995; Ball, Lambell, Reed, & Reid, 2001; Ullman, Herling,

& Sinton, 1996). These studies have brought forward some instances of non-

www.elsevier.com/locate/destud

0142-694X $ - see front matter Design Studies -- (2014) --e--

http://dx.doi.org/10.1016/j.destud.2014.02.006 1� 2014 Elsevier Ltd. All rights reserved.

ss as: Nikander, J. B., et al., The preference effect in design concept evaluation, Design

org/10.1016/j.destud.2014.02.006

mailto:[email protected]

mailto:[email protected]

http://www.elsevier.com/locate/destud

http://dx.doi.org/10.1016/j.destud.2014.02.006

2

Please cite this article in pr

Studies (2014), http://dx.doi

normative behaviour in design decisions (e.g. Cross, 2001; Guindon, 1990;

Jansson & Smith, 1991). However, extant work on decision making in design

contexts has not applied experimental methods and has been mostly qualita-

tive in nature (for instance, Eastman, 1969; Guindon, 1990; Kant, 1985).

This study contributes to a better understanding of the psychological aspects

of design by illustrating the persistence of biases in the concept development

phase of New Product Development (NPD) using an experimental approach.

Specifically we demonstrate how concept evaluations are distorted when de-

signers evaluate a set of concepts including their own designs.

In this paper we first present a model regarding the relationship between

different components in concept evaluation and selection, and the importance

of concept selection is discussed. Next, we present previous research on non-

normative behaviour in design. Third, some relationships between two psy-

chological phenomena, psychological ownership (Pierce, Kostova, & Dirks,

2001, 2003) and the mere ownership bias (Beggan, 1992; Beggan & Brown,

1994), are mapped. Subsequently, we test the hypothesis that designers favour

their own ideas in concept evaluation and selection. We expect that the expe-

rienced concept ownership should have an effect on decision making in

concept evaluation and selection.

Our results show that designers tend to favour their own concepts in concept

evaluation, which has some implications on design practice. We will discuss

this along with some suggestions for further research on the preference effect

in the paper.

1 Concept evaluation and selectionDecision making is an integral part of the NPD process. Important decisions

regarding, for instance, the form and function of the product to be and funding

of development projects need to be made, often with insufficient and inaccu-

rate information (Kihlander, 2011; Legardeur, Boujut, & Tiger, 2010). Design

decisions made in the early phases of NPD, namely in the concept develop-

ment phase (e.g. Ulrich & Eppinger, 2003), are critical for the success of the

product under development. The decisions made in the concept development

phase largely determine the quality, cost, and desirability of the end product

(Asiedu & Gu, 1998). Hence failed concept selection decisions can often be

compensated only with high redesign costs and increased development time

during the later phases of the NPD process (Pahl, Beitz, Feldhusen, &

Grote, 2007).

The concept development phase of the NPD process is typically considered as

a divergenteconvergent activity (cf. Design Council, 2006; Pugh, 1991).

Ideally in this approach, a wide set of alternative product ideas or concepts

are at first generated (divergence), and then evaluated and eliminated in order

to select the best concept or concepts for further development (convergence).

Design Studies Vol -- No. -- Month 2014

ess as: Nikander, J. B., et al., The preference effect in design concept evaluation, Design

.org/10.1016/j.destud.2014.02.006

The preference effect in



As an activity, concept selection has a significant impact on design success

(Mattson & Messac, 2005; Stevanovi�c, Marjanovi�c, & �Storga, 2012).

Some variance exists in NPD literature regarding the terminology used when

discussing concept decisions. Ulrich and Eppinger (2003) define concept selec-

tion as a process in which the concepts are first evaluated and next, the most

promising concepts are selected for further investigation and development.

Cooper (1988) on the other hand denotes this overall step as concept evalua-

tion. In many accounts, such as the Delft Innovation model (Roozenburg,

1977; Roozenburg & Eekels, 1995) evaluation and decision are presented as

subsequent steps in which evaluation is considered to produce an outcome

in terms of perceived value of the design, which then feeds the process of deci-

sion making.

There are many approaches to the arrangement of the evaluation-selection

process. In Pahl et al. (2007) the design team is to first select some solutions

using crude and simple-to-use methods for further scrutiny. Next they carry

out concept evaluation according to elaborate procedures using specific met-

rics, discovering the most suitable solution for the design problem, which is

hence selected. The typical descriptions are analogous to each other. The

term selection used by e.g. Ulrich and Eppinger (2003) is very much similar

to the outcome of the detailed evaluation procedure in Pahl et al. (2007).

Drawing from these, we sketch a normative model in which concept evaluation

is a functional predecessor to concept selection shown in Figure 1. If a concept

is selected according to its performance in the evaluation, the conceptual

design phase is finished and the design team can continue with detailed design.

Rejection of concept can lead to revision or new generation phase.

2 Non-normative behaviour in designDesigners, as a group of professionals, are as susceptible to the same biases

and non-normative behaviour in their work as anybody else. In the existing

body of research, non-normative behaviour has been evident in the solution

generation and evaluation (Ball et al., 2001; Guindon, 1990).

In solution generation, software designers apply opportunistic strategies (Ball

& Ormerod, 1995). They move quickly from high to low levels of abstraction

(Ball & Ormerod, 1995; Guindon, 1990). Another instance of non-normative

behaviour in solution generation is design fixation. Jansson and Smith (1991)

first demonstrated this tendency in how designers fixate on features of example

products presented to them. This influences their designs such that they strik-

ingly resemble the presented examples. Cross (2001) proposed that designers

become attached to their principal ideas, and they try to keep to them as

long as possible, no matter the cost. This, in turn, is similar to the escalation

of commitment identified by Schmidt and Calantone (2002). They showed

design concept evaluation 3


org/10.1016/j.destud.2014.02.006

Figure 1 The normative model of conceptual design and evaluation

4



that managers who have initiated a product development project are more

likely to keep on funding a failing project, than managers who assume leader-

ship later on during the project.

Based on the observations of Kihlander (2011) accumulated over time from a

global R&D department, designers and design teams are vulnerable to a great

multitude of psychological pitfalls discussed by Hammond et al. (1998),

namely anchoring, framing, and confirmation bias. Additionally, Kihlander

(2011) proposes that formal decision making has a minor role in making

concept selections. Hence the formalised concept selection methods proposed

in the literature, such as concept scoring, weighed objectives method, and use-

value analysis (see, e.g., Pahl et al., 2007; Ulrich & Eppinger, 2003) often have

little significance in design practice. This was suggested by Laakso and

Liikkanen (2012) who studied formal methods in everyday design practice

and found designers preferring informal, ad hoc methods throughout the pro-

cess without methodologically following defined process steps.

In fact, L�opez-Mesa and Bylund (2011) report that the use of structured

methods has been somewhat limited in design practice. According to their

findings, the methods are reportedly used as guidelines in concept selection

or they are modified to fit the needs of the designers. If no structured methods

are used, the decisions made in concept selection can exhibit non-normative

behaviour as the designers are not forced to use the same sort of evaluation

strategies for all concepts.

Finally, in their exploration of naturalistic decision making, Ball et al. (2001)

point out that solutions are not explicitly evaluated. The application of design

criteria in evaluation is neither rigorous, nor complete. In fact, according to

the authors (Ball et al., 2001), designers select satisficing, acceptable solutions

instead of optimising the selection due to the complexity of the situation.

3 The ownership effectOwnership is an association between an individual and an entity. People expe-

rience psychological connections between themselves and a multitude of en-

tities of both material and immaterial quality (Beggan, 1992; Beggan &

Brown, 1994; Litwinski, 1947; Pierce et al., 2001, 2003). These associations

may come to being in a multitude of ways. The associations may, for instance,



.org/10.1016/j.destud.2014.02.006




be superficial, where the associations comes to being through mere proximity

of the entity, having control on it or being in physical possession of it. Alter-

natively, the associations may be highly intimate, in which the association is

formed during the creation of the entity or spending extensive time with it

(Furby, 1978; Pierce et al., 2001). Specifically, Shaw, Li, and Olson (2012)

showed that these principles of ownership extend to ideas as well, even for chil-

dren. Pierce et al. (2001) propose that entities becoming targets of ownership

are integrated into the ‘extended self’, as introduced by Belk (1988). As a

result, the objects become inseparable parts of the owner’s psychological

identity.

It is typical for humans to strive for favourable self-presentation (Cialdini & de

Nicholas, 1989) and positive biases in self-perception exist (Taylor & Brown,

1994). People want to be associated with entities that embody positive attri-

butes both for purposes of self-preservation and self-aggrandisement. This ten-

dency may in turn lead to exaggeratedly positive evaluations of objects already

associated with the self e people perceive the objects better than they objec-

tively are. Beggan (1992) found that people do indeed exhibit an evaluative

bias in favour of objects associated with them through various forms of owner-

ship. People are prone to make self-enhancing judgements and the evaluative

bias rises from the need to value parts of the ‘extended self’ higher than other

entities (Beggan, 1992; Nesselroade, Beggan, & Allison, 1999). This bias,

which Beggan called mere ownership effect, is a prevalent phenomenon in a

multitude of situations. Furthermore, Reb and Connolly (2007) showed that

ownership-related non-normative behaviour does not require a factual owner-

ship of the object e a subjective feeling of ownership is sufficient.

A great range of research on ownership exists. For instance, the neural basis of

ownership (Turk, van Bussel, Walter, & Macrae, 2011) and the impact of

ownership on memory and recollective experience (van den Bos,

Cunningham, Conway, & Turk, 2010) along with the effects of perceived

ownership on evaluation (Beggan, 1992; Beggan & Brown, 1994) have been

studied. However, systematic research into the implications of ownership on

decision making, especially in the field of design, has been lacking. Figure 2

illustrates the phases of conceptual design we expect the ownership effect to

have an impact on in terms of the normative model presented earlier.

4 Experimental methodWe designed a two-phase experiment to test the hypothesis that designers

favour their own ideas in concept evaluation and selection. First, the subjects

designed a solution to a bicycle rack problem. After several days, they evalu-

ated a set of solutions to the problem with their own solution either included or

excluded from the set, depending on the experimental condition. Eighteen sub-

jects participated in the experiment. Both quantitative and qualitative data,

including think-aloud protocols (Ericsson & Simon, 1984) were gathered.



org/10.1016/j.destud.2014.02.006

Figure 2 The ownership effect is expected to have an impact on concept evaluation and selection. Here the effect is illustrated in terms of the

normative model presented earlier

6



4.1 SampleOur participants were professional designers from ten Finnish companies.

Multiple design companies were involved in order to sample different organi-

sational cultures and mitigate the possible bias. Although we had no means to

quantify the cultural variability or its essential dimensions, we believe our sam-

ple of companies is big enough to let us focus on the designer psychology.

The sample consisted of eighteen professional designers (N ¼ 18) with a mean

age of 38.5 years (SD¼ 6.6 years), and a mean of 11.6 years (SD¼ 6.6 years) of

relevant work experience. In order to control national culture-related differ-

ences in the extent of self-assertion and related phenomena, only native

Finnish speakers were chosen for the study with the exception of one native

bi-lingual. The sample was predominately male, with only one female partic-

ipant. From now on participants will be referred to as ‘her’ and ‘she’ regardless

of their sex.

4.2 The stimuliWe began stimuli creation by exploring the solution space in an ideation ses-

sion among the research group. We strived for creating concepts that ap-

proached the problem from sufficiently differing points of view and which

were qualitatively divergent. We concluded that six concepts would cover

the different solution possibilities to a satisfactory extent. Hence we refined

six concepts out of the sessions to be used as baseline concepts. The concepts

were preliminary by nature, containing some basic information regarding

the functionality and form of the product. Table 1 illustrates the concepts

that we developed along with the concepts created by the participants.

We asked two experts unfamiliar with the aims of study to check the variation

within the group of concepts and their feasibility in terms of the task. For this,

they used a six-point scale (1 e not valid at all, 6 e a very valid solution). If all

concepts were unfeasible in terms of the design problem, we could not reliably

state whether or not preference of own concepts would arise from some

deviant behaviour, or just from the fact that the baseline concepts were all



.org/10.1016/j.destud.2014.02.006

Table 1 The concepts generated by experimental subjects are illustrated on the two upper rows with the baseline concepts illus-

trated on the two rows beneath

Generated

concepts

Id-lock Touch and lock Star rack Willow branch Shaft

Coin automat Bike hoist Vertical rack (1) Vertical rack (2)

Baseline

concepts

Fan rack Cable wall Bike fence Bike safe Trunk rack

Bike post




of inferior quality. The experts evaluated all baseline concepts to be feasible

solutions to the problem (feasibilityM¼ 4.58, SD¼ 1.0). The experts had pre-

vious experience in concept evaluation and selection both in theory and prac-

tice having worked in academia and design.



org/10.1016/j.destud.2014.02.006

Figure 3 Experimental design. Fir

the first phase of the study, each p

concepts and then evaluated them a

phase, the concepts generated by

8



The experts evaluated the concepts following the idea of creativity assessment

technique (Amabile, 1996) and using relevant dimensions: creativity, novelty,

subjective liking, aesthetic appeal, feasibility, and user-friendliness. Later on,

the same experts were asked to evaluate the nine concepts created by designers

in the experimental group.

4.3 ProcedureThe study consisted of two phases: 1) a concept design phase and 2) a concept

evaluation phase, which were organised approximately seven days apart

(M ¼ 7.06 days, SD ¼ 0.929, range ¼ 3). Each participant did the tasks indi-

vidually without any information given about the other participants. The

concept evaluation phase consisted of three distinct tasks (described in detail

in Section 4.3.2): ranking, scoring, and selection. Figure 3 illustrates the exper-

imental design.

The hypothesis testing involved an experimental setting. The participants were

divided into nominal pairs so that one member of each pair was put in an

experimental group while the other was put in a control group. The allocation

to the groups was random and the participants were unaware of their pair.

Everyone was presented with six baseline concepts and one participant-

created concept (7 concepts altogether). Participants in the experimental group

were presented with their own concept, the control group members were pre-

sented with the concept designed by their pair in the experimental group. The

pairs were matched so that participants from each company were assigned to

both groups on the basis of work experience and age. Each pair was to have as

small a difference as possible in work experience (M ¼ 4.3, SD ¼ 3.5 yrs) and

age (M ¼ 4.7, SD ¼ 3 yrs). Table 2 further illustrates the procedure.

Initially, it was made sure that the task order would not effect the evaluations

as single evaluations could have an effect on the later ranking of the concepts.

st, we designed six baseline concepts to be used in the experiment. Next, experts evaluated these concepts. In

articipant created a solution to the design task. In the second phase, each participant first ranked a group of

ccording to a set of criteria. Finally, the participants selected two concepts from the group. In the post-study

participants were evaluated by experts



.org/10.1016/j.destud.2014.02.006

Table 2 Subjects in the experimental design. One member of each pair was assigned into the experimental group and the other

into the control group

Pair # Subject # Company Age (yrs) Work experience (yrs) Group experimental Control

1 2 A 46 17.5 x1 B 35 10 x

2 4 C 49 24 x5 A 44 15 x

3 3 A 30 2 x10 D 30 2 x

4 7 E 38 12 x9 F 34 8.5 x

5 8 E 38 9 x6 E 33 6.5 x

6 12 E 44 20 x11 E 52 25.5 x

7 13 G 32 9 x14 E 36 10 x

8 15 H 32 6 x16 I 36 15 x

9 17 J 43 14 x18 G 41 15 x




One expert carried out the tasks in the ranking task-scoring task order and the

other in the reverse order. Contradicting the prior assumptions, task order

seemed to have an effect on concept evaluation: if the concepts were first

ranked jointly, the single evaluations did not correspond very well to the

ranks. However, if the concepts were ranked following the single evaluations,

the ranks and points given in the evaluations were very similar (the concept

with most points was ranked as best and so forth). To maximise information,

the former task order was selected. Hence the concepts were first ranked in a

joint fashion, and later scored one at a time.

4.3.1 Phase 1 e concept designIn the first phase, participants were asked to design a solution to rack bicycles

in an urban setting. This task was chosen on the basis of the expected familiar-

ity of the problem space to everyone and the relatively wide range of different

possible solutions to the task. Bicycle related tasks have also been applied pre-

viously in design research (Akin & Lin, 1995; Goldschmidt, 1995; Jansson &

Smith, 1991).

Phase 1 was carried out remotely. The participants were informed about the

general aims of the study and the procedure via e-mail according to a script.

Next, they were allowed to ask further questions from the researcher via phone

or Skype call. However, communication was kept to a minimum in order to

keep the level of information regarding the task as identical as possible for

all participants. The participants did not make questions outside the materials

to be used and time available. The participants completed the first phase at

their own offices.



org/10.1016/j.destud.2014.02.006

10



The participants were given 20 min to design a solution to the problem. The

solution was supposed to contain a sketch of structure of the solution and

additional information regarding the functionality and materials of the

concept. To conclude, the participants were required to photograph or scan

their solutions and send them to the researcher electronically. Each session

lasted for 30 min maximum (M ¼ 27.3, SD ¼ 4 min).

4.3.2 Phase 2 e concept evaluationThe second phase took place in the presence of the experimenter, usually at the

participant’s office. The sessions were recorded using a portable audio recorder.

The participants were asked to individually evaluate the proposed solutions to

the design problem used in the first phase, while thinking aloud. Figure 4 illus-

trates how the concepts were distributed for evaluation in each pair.

This arrangement meant that subjects #1 and #2 were presented the six con-

cepts of our design, and the concept developed by subject #2. In the next

pair, subjects #3 and #4 were presented the same six predesigned concepts

and the concept developed by subject #4 and so forth. The concepts were

shown in a counter-balanced order across the subjects using the Latin squares

method to eliminate presentation order effects.

To control the effects of personal visual skills and preferences related to the use

of colour and different drawing styles, all concepts from the experimental

group were redrawn into a visually uniform format by the third author, who

also visualised the baseline concepts.

Prior to the evaluations, the participants practised thinking aloud with three

practice tasks (Atman & Bursic, 1998; Chi, 1997; Ericsson & Simon, 1984).

Concept presentation followed the practice. The concepts were provided one

at a time and the subjects were given 1 min per concept to familiarise them-

selves with each.

Next, the subjects were to rank the concepts from 1 to 7, one being the best and

seven the worst. This was the ranking task. Having done this, the participants

were asked to evaluate the concepts one at a time on 3 dimensions: usability,

looks and feasibility. This scoring task was performed using a six-point scale

ranging from ‘extremely bad’ to ‘extremely good’. Finally, in the selection

tasks, the subjects were asked to choose two concepts, which they would pro-

pose to be developed further by the City of Helsinki City Planning Depart-

ment. All of these tasks were to be carried out while thinking aloud. The

second phase lasted for a maximum of 35 min (M ¼ 25.2 min, SD ¼ 6).

4.4 AnalysisThe analysis was divided into two supporting parts: qualitative and quantita-

tive analysis. Using protocol analysis (Chi, 1997; Ericsson & Simon, 1984), we



.org/10.1016/j.destud.2014.02.006

Figure 4 Distribution of concepts for evaluation in the experiment. A pair consisting of subjects A and B create a concept in the first phase of the

experiment. In the second phase, both subjects scrutinise the same set of concepts




developed coding schemes to emphasise two phenomena: 1) the use of posses-

sive nouns linked to psychological ownership and 2) the use of evaluation

criteria, related to justifying the dimensions used in the scoring task. The seg-

ments containing possessive nouns related to the participants’ own designs

were identified and coded in all of the tasks (cf. operationalisation of owner-

ship effect in van Dyne & Pierce, 2004). Each subsequent mention related to

the participants’ designs were coded as well. Next, the nature of each segment

(e.g., whether or not the possessive nouns were related to the familiarity or

preference of a concept) was analysed in more depth.

The coding scheme regarding the criteria was developed by the first and the

second author bottom-up according to emergent patterns in the data. Three

different categories of criteria were identified: explicit, implicit and multi-

occurrence implicit criteria (MOIC). The explicit criteria were gathered from

utterances in which the participants clearly stated that they were evaluating

a concept according to a criterion. Table 3 shows two instances of how explicit

criteria were discovered from the data. Notice, that each subsequent occur-

rence of explicit criteria was coded as well. After agreeing upon the codes

and their usage, the first author carried out the coding. The second author

randomly inspected parts of the coded protocol and through discussion

resolved any arising discrepancies without measuring inter-rater agreement.



org/10.1016/j.destud.2014.02.006

Table 3 This excerpt shows how subject #6 established two explicit criteria (space saving e SPA, and safety e SFA) for the

ranking task

Line # Criterion Segment

81 So I’d set two criteria to be used here82 SAF/2 One would be safety from burglars83 SPA/2 And the second [criterion] would be that it

wouldn’t take a lot of space on the street

Table 4 An excerpt from the c

the concepts are ranked. On lin

on the MOIC

Line #

140141142143

12



Implicit criteria were used in a more indirect manner, usually occurring as jus-

tifications for individual ranking decisions (see Table 4 for an example). MOIC

is a category of implicit criteria in which the criterion was systematically used

multiple times throughout the evaluation. MOIC were used in a very similar

manner to explicit criteria as justifications for decisions, but never established

explicitly. We required that the criterion was used at least three times in order

to grant the status of MOIC. Implicit criteria were identified in order to map

out the more dynamic aspects of behaviour during concept evaluation (cf.

detection of implicit decomposition in Liikkanen & Perttula, 2009).

The differences in evaluation between the experimental and control group were

analysed using the Wilcoxon signed-ranks test performed in SPSS software.

5 ResultsThe concepts developed by the subjects during the first phase varied signifi-

cantly. All subjects produced a valid solution to the task. The mean validity

score from expert evaluation was 3.69 (SD ¼ 1.1; scale 1e7). As was the

case with the baseline concepts, on average, the concepts were evaluated valid

and feasible solutions to the design problem. Some subjects provided highly

visual renditions of their ideas whereas others provided highly verbose descrip-

tions of the concept’s functionality with little attention paid to the visual as-

pects. All participants in the experimental group identified their own

concepts, which provides a manipulation check for the experimental

procedure.

5.1 Quantitative findingsThe participants showed a preference for self-initiated designs in two of the

three tasks used. The concept-scoring task yielded a statistically significant

oded protocol of subject #4. Here the MOIC used on lines 141 and 142 have an effect on the way

e 143 the subject ranked the fan rack better than the frame rack according to their performance

Criterion Segment

Actually these both are goodSPA/1 This [cable wall] solution saves spaceSPA/1 Whereas this frame rack takes a lot of space

So I’ll put them [concepts] into the following order



.org/10.1016/j.destud.2014.02.006

Table 5 Wilcoxon signed-rank

sions of evaluation

ZP




preference of participants’ own concepts. In the ranking task, a slight prefer-

ence was evident, but failed to reach statistically significant levels. The selec-

tion task did not yield any clear preference.

In the concept-scoring task, the overall mean concept value aggregated from

the three sub-scores was 11.35 points (SD ¼ 2.80). The mean value of self-

initiated designs in the experimental group was 12.13 (SD ¼ 1.73) versus a

mean value of 10.75 (SD ¼ 3.28) in the control condition. The participants

showed a great variance in evaluation styles, some giving lower than mean

values (e.g.,Msubject 7¼ 8.71) to all concepts meanwhile some giving extremely

high values to all concepts (e.g., Msubject 1 ¼ 14.43). Thus, the evaluation style

was controlled by ipsatisation (Chan, 2003; Fischer, 2004). This showed a sta-

tistically significant difference between the groups (Z ¼ �1.960, p < 0.05),

showing a greater mean value for self-initiated design. None of the scores

on the individual dimensions showed statistically significant differences in

evaluations (shown in Table 5). Figure 5 shows the differences between the ip-

satisised scores in the two experimental conditions.

In the concept-ranking task, the participants in the experimental group ranked

their own concepts slightly higher than their counterparts in the control group

with amean ranking of 4.63 (SD¼ 1.06) versus ameanof 3.38 (SD¼ 1.69) in the

control group. However, these differences were not statistically significant

(Z ¼ �1.529, p ¼ 0.126). As we perceived considerable variation between the

pairs regarding the effect, we decided to investigate the volatility of the effect us-

ing a jack kniving procedure. By systematically removing each pair, one by one,

from the analysis, wewitnessed that the p-value droppedbelow0.1 in four out of

nine instances, reaching significance p � 0.05 two times. However, with an un-

known effect size, the statistical power of the current sample is difficult to judge.

In the concept selection task, three self-generated concepts were chosen for

further development by participants in the experimental group (N ¼ 9). Addi-

tionally, one participant from the control group chose a concept designed by a

participant in the experimental group. Additionally, two baseline concepts

succeeded very well in the evaluations (illustrated in Table 1). Eight subjects

selected the Bike Post and seven selected the Bike Fence for further develop-

ment (note that some subjects may have chosen both). Both concepts were

praised as novel and interesting solutions to the given problem. However, in

the other tasks, no concept was clearly superior to others.

s test statistics for the difference in evaluation of the self-generated concepts on the three dimen-

Usability Looks Feasibility

�1.400 �1.820 �1.5400.161 0.069 0.123



org/10.1016/j.destud.2014.02.006

Figure 5 Scores of the self-initiated design on the three evaluation dimensions and their sum. All values ipsatisised

14



A connection between the ranking evaluations and concept selection was

evident. The concepts selected for further development had a statistically

significantly higher ranking value (t ¼ 10.189, df ¼ 15, p < 0.001) than the ex-

pected mean ranking value of a concept (M ¼ 4).

5.2 Verbal protocol dataDuring the evaluation phase, eighteen protocols were collected. The protocols

were segmented so that each segment was to comprise a ‘complete thought.’

This follows the approach proposed by Trickett and Trafton (2009), in which

‘complete thought’ refers to a clause containing a subject and a predicate. On

average, over two hundred segments were identified per protocol, but there

was a great variance between the subjects (M ¼ 234.4, SD ¼ 82.2 segments)

giving a hint of different levels of verbosity. Statements regarding the practice

tasks and the instructions were omitted from the analysis.

5.2.1 Evaluation criteriaThe participants used a great variety of explicit and implicit criteria according

to which the concepts were ranked (shown in Table 6). 121 segments that con-

tained some use of criteria were identified in the data. All participants used

some set of criteria in the evaluations, but differences in their use were com-

mon. Altogether, the mean amount of criteria used per subject was 7.5

(SD ¼ 3.35). The means for explicit and implicit criteria and MOIC, respec-

tively, were as follows: Mexplicit ¼ 2.75 (SDexplicit ¼ 2.98), Mimplicit ¼ 2.69

(SDimplicit ¼ 2.02), and MMOIC ¼ 2.06 (SDMOIC ¼ 2.32).



.org/10.1016/j.destud.2014.02.006

Table 6 The use of explicit, multi-occurrence implicit and implicit criteria during ranking task in all protocols

Criterion Code Explicit MOIC Implicit Total %

Space saving SPA 8 17 5 30 24.79Novelty value NOV 14 3 7 24 19.83Feasibility FEA 7 0 10 17 14.05Safety SAF 4 3 5 12 9.92Price PRI 2 0 7 9 7.44Usability USA 4 0 5 9 7.44Looks LOO 1 6 2 9 7.44Simplicity SIM 2 4 0 6 4.96Meets requirements MRQ 1 0 1 2 1.65Social context SOC 1 0 1 2 1.65Design quality DSQ 1 0 0 1 0.83Total 45 33 43 121 100% 37.19 27.27 35.54 100 100




We also observed that the majority of the criteria reoccurring between subjects

as domain-general, or task non-specific. Feasibility, novelty, price, and usabil-

ity were mentioned by at least half of participants. Space saving was the single

task-specific criterion repeatedly used by the subjects. We have reported results

regarding the evaluation criteria separately at length (see Nikander,

Liikkanen, & Laakso, 2013).

5.2.2 Evaluation stylesThe subjects exhibited a variety of approaches to concept evaluation in the

experimental sessions. In ranking tasks, some participants evaluated the con-

cepts in a very analytic manner, thoroughly contemplating the features of the

concepts and comparing them to existing solutions in depth. For instance one

participant explicitly stated that she would use pairwise comparisons and pro-

ceeded by doing just that in a clear demonstration of an evaluation strategy.

Others made evaluations in a more intuitive manner. The concepts were merely

glanced at and the decisionsweremade swiftly. In terms of criteria use, themore

thorough evaluations exhibited an increasing dependency on criteria.

The differences in evaluation styles persisted in the scoring task in which some

subjects carefully contemplated the scores for each given dimension whereas

some scored the concepts quickly without further deliberation. Due to the na-

ture of the task, most evaluations followed a uniform pattern in which each

concept was evaluated according to the dimensions given.

The behaviour in the selection task was again variable: many a participant

made the decisions quickly by referring to the previous tasks whereas some

went to great lengths to determine the concepts best fit for further develop-

ment. However, some of the participants used more analytic means where

they removed concepts from the evaluation according to their performance

on some dimension. Here concepts were eliminated iteratively from the evalu-

ation if some negative aspect was identified.



org/10.1016/j.destud.2014.02.006

Table 7 A translated excerpt

shown

Line #

747782

16



The selection task yielded some self-contradictory results. Some participants

had evaluated concepts fairly low in both the ranking and scoring tasks.

Regardless, they still chose the very same concepts for further development

due to some feature of the concepts. Subject #8, for instance, stated that the

Bike Post baseline concept was ‘[it’s a] crazy [idea], but it’s so mind-

boggling both as an idea and by looks’ and selected it for further development.

The results show that designers employ a great variety of different evaluation

styles and criteria when provided with a quasi-structured evaluation method

(cf. methods in Ulrich & Eppinger, 2003). In summary, the styles may be

described as having been either analytic or opportunistic. Some participants

applied a highly analytic approach, whereas some stuck to a quick and opportu-

nistic evaluation style, sometimes applying novel methods. In general, opportu-

nistic evaluations were slightly more common in the experiment, but no

dominating style emergedduring the experiment.Furthermore, even the analytic

evaluations were erratic at times and switches in evaluation style were common.

5.2.3 Other findingsPossessive behaviour was not discovered in the protocol analysis. Only 26 seg-

ments (a mere 0.7% of all segments) from six participants in the evaluation

phase were identified as possessive. Five of these participants were in the

experimental group and one was in the control group. The identified utter-

ances usually had to do with the familiarity of one’s own concepts and some

explicitly stated that they preferred their own concepts. Subject #7, for

instance, identified her own concept during the ranking task and reported

the following: ‘I’ll go with [.] I need to rank my own idea as # 1’ (see

Table 7). Furthermore, two designers with extensive experience in their field

exhibited reluctance to include their own concepts in the evaluation. Finally,

one subject in the control group clearly stated that two predesigned concepts

resembled her own idea significantly. Interestingly, she ranked these two con-

cepts highest in the ranking task. A similar case was discussed in the previous

section where a participant thought two concepts were actually improvements

over her own idea, resulting in a selection of those two concepts.

We also discovered a phenomenon we call ‘evaluation neglect.’ Four out of

nine participants in the experimental group expressed that they were highly

familiar with their own concepts and did not discuss them in the same depth

as other concepts. Expressions used in these cases ranged from ‘I’m familiar

from the coded protocol of subject #7. Here, a clear preference of the subject’s own concepts is

Concept Segment

Own And, somehow, my own ideas feel really goodOwn And I’ll go with, I need to rank my own idea as # 1Own I guess that my own concepts will be the best ones now



.org/10.1016/j.destud.2014.02.006




with this idea’ (subject #4) to ‘I know how this is supposed to work’ (subject

#7). Furthermore, participant #7 stated that she just ‘had to’ choose her

own concept, regardless of the other concepts. In addition, one participant

thought that her own idea was divided into two distinct concepts, and she

ended up choosing a concept she deemed to be an ‘upgraded’ idea of her

own along with one predesigned concept.

Some subjects had trouble in verbalising their thoughts while carrying out the

tasks (as suggested by the great variance of segments in the protocols). One

subject reported that the concepts’ visual features were a driving factor in

her evaluations. She had difficulty in verbalising her perceptual experience dur-

ing the selection process. However, the majority of participants were able to

verbalise their thoughts during the evaluations.

6 DiscussionThis study set out to investigate the psychological aspects of concept selection.

An experimental setting was developed in order to study if designers form a

sense of psychological ownership towards their own designs and if this leads

to systematic differences in their evaluations. Measurements focussing on

both evaluation and decision making were used. Designers seemed to prefer

their own concepts in two evaluation tasks. However, the protocol data did

not reveal whether the preference was due to psychological ownership (as in

Beggan, 1992; Beggan & Brown, 1994), as our planned measurement failed

to capture the desired data. Henceforth, the phenomenon identified in the

data is referred to as the preference effect.

The discovery of a preference effect is the strongest support for our hypothesis

of non-normative behaviour in design decision making. However, it is not the

only one. We also observed self-contradictory behaviour between ranking and

evaluation tasks versus decision task, opportunistic evaluation styles, and

evaluation neglect e all instances of non-normative behaviour. In contrast,

the lack of experimental effect in the selection task and the coherence between

ranking and selection tasks (see Sections 5.1 and 5.2.2) are against our

hypothesis.

The preference of one’s own ideas is evident in both the quantitative and the

qualitative data. Throughout the different tasks, self-generated ideas seemed

to be preferred to some extent, the preference being statistically significant

in the scoring task. Interestingly, the effect was not statistically significant in

the ranking or selection tasks, which are less structured by nature.

Regarding the normative model of conceptual design (in Figure 1), our find-

ings are ambiguous. In the experiment, the decisions were associated with eval-

uations in the predicted way, although the relationship was not very strong.

However, as the effect was not apparent in the selection task, it is hard to



org/10.1016/j.destud.2014.02.006

18



determine how serious the issue is. Nevertheless, the discovery of the prefer-

ence effect demonstrates the risks of decision making under biased evaluation

results.

6.1 Accounts for the preference effectThere are several possible explanations for the observed preference effect. We

will consider explanations based on a motivational account, information pro-

cessing perspective, and personal values perspective.

6.1.1 The motivational account: the ownership effectIf the participants formed a relationship of ownership towards their own de-

signs and started associating themselves with their concepts (as suggested by

Pierce et al., 2001), these findings might be accountable to the mere ownership

effect identified by Beggan and Brown (1994). However, some issues regarding

the experimental method need to be discussed in detail before making the

inference.

First, the emergence of psychological ownership was operationalised as the use

of possessive nouns (van Dyne & Pierce, 2004) when subjects were presented

their own concepts and asked to evaluate them. Protocol analysis and the as-

sumptions made regarding the use of possessive nouns have been developed in

an Anglo-American framework. That culture is different from that of Finland.

It has been acknowledged that Finns are traditionally modest by nature and

this may have an impact on their verbalisations (Nishimura, Nevgi, & Tella,

2008), resulting in lesser amounts of self-asserting behaviour. Thus, perhaps

the use of possessive nouns is not a suitable operationalisation of psycholog-

ical ownership in the given cultural framework.

Second, as a process the emergence of psychological ownership might be too

implicit to be verbalised, making it impossible to identify by means of protocol

analysis (Wilson, 1994). Furthermore, a significant issue regarding the exper-

imental method was the short duration of the design phase. Many participants

reported that 20 min was too short a time to properly contemplate the problem

in depth. This, for its part, might have resulted in poor commitment to one’s

own concepts and thus, to the absence of possessive nouns in the qualitative

data. Given the short time frame, the participants could not invest themselves

properly to the concepts, which is a crucial process to the emergence of psycho-

logical ownership (Pierce et al., 2001, 2003).

Third, due to the fact that the statistically significant result was identified in the

concept-scoring task, it is important to discuss the validity of the dimensions

used in the evaluation. The evaluation criteria arising from the ranking task

give some justification for the dimensions used in the scoring task as 29 per

cent of the identified criteria were identical to those used in the scoring task

without a cue. In terms of the experiment, some adjustments could have



.org/10.1016/j.destud.2014.02.006




been made to the dimensions used in evaluations. The novelty value criterion

might have been very well incorporated into the evaluation (as it was in the

expert evaluations) due to its relative ambiguity. The space saving criterion,

however, is highly task specific by nature and could have had a biasing effect

in the evaluations, making some concepts more preferable than others.

Fourth, the visual modification used in the experiment might have, in fact, had

a significant effect on the strength of the association between the individual

and the concept. Some dimensions of the concept their designers might have

thought essential, may have been left out. Moreover, the modification might

have made the concept visually less appealing for the participants. In fact,

some participants reported that somebody had copied their ideas or been

‘fiddling around’ with their concepts, referring to our baseline concepts. Sec-

ond, the participants might have felt denouncing their own concepts is socially

desirable, which might have diminished the effect. Some factors, however,

might have had an effect in both directions. For instance, the participants

might have become aware of the experimental hypothesis and changed their

behaviour accordingly. This hypothesis awareness might have strengthened

or diminished the participants’ tendency to favour their own concepts. This,

however, depends on the participant. Second, along similar lines, mere aware-

ness of participation and the novelty of taking part in an experiment may have

influenced the way the participants behave in the experiment.

Overall, the experimental method could not exhaustively identify a causal

connection between emergence of psychological ownership and the observed

phenomenon. However, some systematic connection between having designed

a concept to a design problem and preference of one’s own concepts is evident

in the data.

6.1.2 The information processing perspectiveThe motivational account is not, however, the only possible explanation for

the phenomena. Discussing the limitations of memory and information pro-

cessing in design, Ball, Maskill, and Ormerod (1998) point out that recognition

of a salient solution idea sets up inhibitory activations within long-term mem-

ory. This inhibitory activation might by itself block deeper processing of other

concepts generated for the same design problem. Hence self-generated con-

cepts are 1) more available, salient and familiar, and 2) easier to process.

Furthermore, more fluently processed information is taken to be more truth-

ful, pleasing and acceptable than other processed information (Alter &

Oppenheimer, 2008; Shah & Oppenheimer, 2007). Accordingly, the concept

most familiar to the participants is evaluated higher.

Similar to the above, the behaviour could have resulted merely from informa-

tion asymmetry. Participants may have contemplated some parts of the solu-

tion space more than others, making them better informed about the possible



org/10.1016/j.destud.2014.02.006

20



pros and cons of the given concepts and thus biasing them towards solutions in

this solution category. However, this might have a reversed effect as well mak-

ing the negative aspects of solutions more easily identifiable. Some subjects ex-

pressed that they did not quite understand some of the concepts, perhaps

biasing them towards ideas they were better informed about. The effects of

asymmetric quantities of information are closely related to the confirmatory

bias where individuals heed only information that accords to their preconcep-

tions (Stanovich, 2003). In the case of concept selection, it would be expected

that designers would only take the positive aspects of their own concepts into

account meanwhile belittling the positive aspects of other concepts. However,

no such behaviour was observed in the protocol data.

6.1.3 Personal valuesThe preference effect could have resulted from designers’ intrinsic values. In-

dustrial designers are highly dependent on value-related reasoning in making

design decisions (Holm, 2006, pp. 279e282). This notion also extends to the

evaluation of concepts: industrial designers favour ideas that reflect their

own values (Holm, 2006, pp. 325e326, 342). Naturally, designers’ own ideas

(and designs) should somehow reflect their own inner values e what they

see as important and what should have been taken into account in any given

design solution e leading to preference of one’s own ideas. Especially, this

value-related phenomenon could explain the instances where the subjects

preferred concepts they took to be similar to their own.

6.2 Evaluation stylesTraditional criteria associated with designers’ values, such as novelty, feasi-

bility, and usability were most common explicit criteria along with the more

task-specific space saving criterion. Surprisingly, in contrast to prior beliefs

(e.g., Holm, 2006), aesthetics was not a significant criterion in the ranking

task. The same criteria (novelty value, feasibility, and usability) shared amajor-

ity in implicit criteriause aswell.However, their dominancewasnot as clear as in

explicit use. The use of explicit criteria was associated with normative decision

making, during which subjects first establish an evaluation strategy and make

their decisions according to the given strategy. Although some participants

clearly stated they would be using a given set of explicit criteria in their evalua-

tions, they ended up using implicit criteria or MOIC in their evaluations. Usu-

ally these cases started with the participant using one of the more popular

criteria (e.g. novelty value) as a starting point for the evaluation, but ended

up using some of the highly task-specific dimensions (such as safety or space

saving) as criteria. These deviations from explicit criteria give hints of the perva-

sive nature of task-specific criteria when designers are not given, or apparently

following, any structured method. It may be in fact due to the explicit criteria

getting overridden by features evident in the concepts. When given a consider-

able amount of freedom in the evaluations, a great divergence in the evaluations

became evident in the data where some concepts were scrutinised in different



.org/10.1016/j.destud.2014.02.006




quantities andbyusing different approaches to evaluation.The great divergence

in evaluation strategies and criteria, alongwith internal conflicts and switches in

evaluation style, supports the use of structured methods in concept selection.

6.3 Implications on design practiceAsymmetric quantities of information or designers’ values might very well be

responsible for the preference effect but, functionally, their effects are identical

to those of psychological ownership. At a first glance, one could conclude that

some controlling measures may need to be taken when carrying out concept

selection. A control measure could be, for instance, the widespread use of

structured concept selection methods. This, unfortunately, might not be suffi-

cient due to apparent inconsistencies between concept evaluation and selection

in our data.

The most alarming finding in terms of design practice is that the statistically

significant difference occurred in the most well-defined task (scoring task).

One would expect that when given explicit criteria according to which the con-

cepts should be evaluated, the evaluation should be as analytic (or normative)

by nature as possible in the given task space. The fact that the preference effect

was strongest in this mode of evaluation implies that simple, formal concept

evaluation methods might be susceptible to the same biased decision making

as observed in the experiment. A natural conclusion could be the use of a

more rigorous concept selection method, such as s-Pareto frontier selection

(Mattson & Messac, 2005) or Electre II (Vinodh & Girubha, 2011). The prob-

lem with these methods is their innate complexity and time-consuming nature,

requiring training and notable time investment. Hence designers need to either

balance between the susceptibility to biases of the simple methods and the

innate complexity of the more refined methods or, alternatively, come up

with novel solutions to address this issue.

On the other hand, the consistency between concept ranking and evaluations

was a resounding finding. The concepts ranked highest were consistently

selected for further development. If not implying rationality in the sense the

concept is used in psychology (see, e.g., Gigerenzer, 2006), it still indicates

that results from different phases in concept evaluation and selection do indeed

correlate. Although the present study cannot systematically show that the cor-

relation arises from the ranking task carried out prior to concept selection, the

finding implies that the selections are influenced by the rankings and selections

are not made completely haphazardly.

We present four alternative approaches to mitigating the preference effect. 1)

A simple remedy could be preventing designers from evaluating concepts alto-

gether. Naturally, this would have a twofold effect on the evaluation as the de-

signers bring valuable knowledge of the design problem into the evaluations as

well, while still being biased in their decisions. 2) Designers should recuse



org/10.1016/j.destud.2014.02.006

22



themselves from the evaluation when presented with their own concept. Hence

only other people should evaluate their concepts. 3) An alternative measure

could be developing simple structured methods in a way that they would

take the preference effect into account, thus, being easier to use than the

more rigorous methods while still controlling the effect to some extent. The to-

tal score designers can give to their own ideas could be, for instance, restricted

(as is the case in some organisations at the time of writing). 4) Finally, de-

signers should be made aware of the preference effect in order to prevent

blatant failures in design. Kihlander (2011, p. 65) points out that if designers

have sufficient awareness and meta-knowledge of possible psychological traps

and the dependencies in concept selection, similar risks (among other potential

traps in design) might be mitigated.

No matter the approach chosen to mitigate the effect, one needs to keep in

mind the complex process, by which the concept decisions come to being.

As Kihlander (2011, p. 23) suggested, the decisions do not come into being

discretely in any meetings. Instead, the decisions emerge dynamically in the in-

teractions between the team members during the design process. Accordingly,

individual designers are seldom responsible for selecting a concept to be devel-

oped further. Hence the observed phenomenon does not necessitate that de-

signers’ own concepts would be always selected for further development in

design practice.

6.4 Limitations and future workThe preference effect, among other findings, gives a hint about the pervasive

nature of non-normative behaviour in concept evaluation and selection and

suggest that future research on the subject is necessary. Longitudinal studies

with relevant projects with greater personal investment from the participants

would be preferred. Replication attempts should utilise a similar coding

scheme and include a measure of inter-rater reliability. Future work should

also take into consideration some limitations of the current experimental

setting and how the changes in the design could improve upon it.

As presented in Section 5.1, the preference effect is somewhat volatile and its

effect size still unknown. The implication is that the effect in the current exper-

iment was moderate, which may be due to the visual modifications of the con-

cepts carried out as an experimental control and social desirability.

Furthermore, the effects of task order need to be heeded in future work as

evaluation-ranking task order might provide completely different results. A

further possible source of bias in the data relates to the question of how seri-

ously the participants took the design task. After all, the negative ramifications

of creating or selecting a suboptimal design were minimal and this might have

affected their behaviour. Based on observing the participants in the evaluation

session and looking at the evaluation transcripts, we have a reason to believe

that the participants took the task sufficiently seriously. First, in idea



.org/10.1016/j.destud.2014.02.006




generation the generated concepts were valid solutions to the problem, as illus-

trated by the expert evaluations. Second, in the evaluation phase, the partici-

pants paid due attention to all concepts and nobody commented that their

behaviour was unlike what they would do in their work practice. Finally, as

shown in the expert evaluations, the participants provided us with valid solu-

tions to the design problem. We feel that this speaks strongly in favour of our

earlier position that the participants took the task seriously.

Earlier, we proposed that more widespread application of structured methods

could be a possible remedy to the preference effect identified. However, the

problem regarding non-normative behaviour remains: in the end structured

methods are to be used by human beings. Hence some research regarding the

applicability of structured methods remains to be done in terms of human

behaviour. Additionally, some inconsistencies between concept evaluation

and selection were apparent in our data, and thus, we propose future research

on these mismatches between different phases in concept evaluation and selec-

tion. We identified a great divergence of criteria and evaluation styles and sug-

gested the use of structured methods as a possible controlling measure.

Another possible subject of future research is that of the effects of the amount

of concepts in the evaluation. In their study, Hsee, Loewenstein, Blount, and

Bazerman (1999) identified a preference reversal between joint evaluation and

separate evaluation of options. When carrying out single evaluations, people

end up with remarkably divergent ways of evaluation than in joint-

evaluation conditions and we know concepts are nearly always assessed in a

joint fashion.

A final suggestion for future research is that of biases against creativity.

Mueller, Melwani, and Goncalo (2012) identified a tendency for people to

be biased against creative ideas. Furthermore, creative ideas are sometimes

winnowed by purpose in organisations (Amabile, 1998). Regardless of explic-

itly supporting creative thinking and ‘out-of-the-box’ problem solving, crea-

tive ideas are shunned on. This bias could have grave implications on design

practice when innovative and novel ideas are not supported. Hence its impact

on decision making in concept selection should be mapped.

7 ConclusionAs a conclusion, this study identified a clear preference for the designers’ own

concepts in concept evaluation. Three alternative causes for the preference

were discussed: psychological ownership, designers’ values and asymmetric

quantities of information. Regardless of the underlying cause, the implications

on design are significant. No single controlling measure might be able to

completely control the effect, but with sufficient compromises a satisfactory so-

lution might be reached. Future research on non-normative behaviour in

design was suggested to further map out how biases and deviations from the



org/10.1016/j.destud.2014.02.006

24



norm affect the design process and concept selection. Further research on the

subject may give hints of how structured methods should be developed and

how design practice should be structured.

AcknowledgementsWe would like to express our gratitude to the interviewed designers and their

organizations for volunteering to take part in the research, Professor Kalevi

Ekman for help in finding the informants, and the Technology Industries of

Finland Centennial Foundation for funding the research.

ReferencesAkin, €O., & Lin, C. (1995). Design protocol data and novel design decision.

Design Studies, 16(2), 211e236.

Alter, A. L., & Oppenheimer, D. M. (2008). Easy on the mind, easy on the wallet:the roles of familiarity and processing fluency in valuation judgments. Psycho-nomic Bulletin & Review, 15(5), 985e990.

Amabile, T. M. (1996). Creativity in context. Boulder, CO: Westview Press.Amabile, T. M. (1998). How to kill creativity. Harvard Business Review,

76(August), 77e87.Asiedu, Y., & Gu, P. (1998). Product life cycle cost analysis: state of the art re-

view. International Journal of Production Research, 36(4), 883e908.Atman, C. J., & Bursic, K. M. (1998). Verbal protocol analysis as a method to

document engineering student design processes. Journal of Engineering Educa-tion, 87, 121e132.

Ball, L. J., Lambell, N. J., Reed, S. E., & Reid, F. J. M. (2001). The exploration ofsolution options in design: a ‘naturalistic decision making’ perspective. In

P. Lloyd, & H. Christiaans (Eds.), Designing in context (pp. 79e93). Delft,The Netherlands: Delft University Press.

Ball, L. J., Maskill, L., & Ormerod, T. C. (1998). Satisficing in engineering design:

causes, consequences and implications for design support. Automation in Con-struction, 7, 213e227.

Ball, L. J., & Ormerod, T. C. (1995). Structured and opportunistic processing in

design: a critical discussion. International Journal of Humanecomputer Studies,43, 131e151.

Beggan, J. K. (1992). On the social nature of nonsocial perception: the mereownership effect. Journal of Personality and Social Psychology, 62(2),

229e237.Beggan, J. K., & Brown, E. M. (1994). Association as a psychological justification

for ownership. The Journal of Psychology, 128(4), 365e380.Belk, R. W. (1988). Possessions and the extended self. The Journal of Consumer

Research, 15(2), 139e168.van den Bos, M., Cunningham, S. J., Conway, M. A., & Turk, D. J. (2010). Mine

to remember: the impact of ownership on recollective experience. QuarterlyJournal of Experimental Psychology, 63, 1065e1071.

Chan, W. (2003). Analyzing ipsative data in psychological research. Behaviorme-

trika, 30(1), 99e121.Chi, M. T. (1997). Quantifying qualitative analyses of verbal data: a practical

guide. The Journal of the Learning Sciences, 6(3), 271e315.Cialdini, R. B., & de Nicholas, M. E. (1989). Self-presentation by association.

Journal of Personality and Social Psychology, 57(4), 626e631.



.org/10.1016/j.destud.2014.02.006

http://refhub.elsevier.com/S0142-694X(14)00030-1/sref1



























































Cooper, R. C. (1988). Predevelopment activities determine new product success.Industrial Marketing Management, 17(2), 237e248.

Cross, N. (2001). Design cognition: results from protocol and other empiricalstudies of design. In C. M. Eastman, W. M. McCracken, &

W. C. Newstetter (Eds.), Design knowing and learning: Cognition in design ed-ucation (pp. 79e103). Oxford, UK: Elsevier Science Ltd.

Design Council. (2006). Double diamond design process. Available from. http://

www.designcouncil.org.uk/webdav/harmonise?Page/@id.53&Session/@id.D_4jaHtwk0Hj7ve5elIToe&Document/@id.10149.

van Dyne, L., & Pierce, J. L. (2004). Psychological ownership and feelings of

possession: three field studies predicting employee attitudes and organizationalcitizenship behavior. Journal of Organizational Behavior, 25, 439e459.

Eastman, C. M. (1969). Cognitive processes and ill-defined problems: a case study

from design. In Proceedings of the international joint conference on artificial in-telligence: ICJAI’69 (pp. 669e690).

Ericsson, K. A., & Simon, H. A. (1984). Protocol analysis: Verbal reports as data.Cambridge, MA: MIT Press.

Fischer, R. (2004). Standardization to account for cross-cultural response bias: aclassification of score adjustment procedures and review of research in JCCP.Journal of Cross-Cultural Psychology, 35(3), 263e282.

Furby, L. (1978). Possession in humans: an exploratory study of its meaning andmotivation. Social Behavior and Personality, 6(1), 49e65.

Gigerenzer, G. (2006). Bounded and rational. In R. B. Stainton (Ed.), Contempo-

rary debates in cognitive science (pp. 115e136). Malden, MA: Blackwell Pub-lishing Ltd.

Goldschmidt, G. (1995). The designer as a team of one. Design Studies, 16(2),

189e209.Guindon, R. (1990). Designing the design process: exploiting opportunistic

thoughts. HumaneComputer Interaction, 5, 305e344.Hammond, J. S., Keeney, R. L., & Raiffa, H. (1998). The hidden traps in decision

making. Harvard Business Review, 76(SeptembereOctober), 47e58.Holm, I. (2006). Ideas and beliefs in architecture and industrial design: How atti-

tudes, orientations, and underlying assumptions shape the built environment.

(PhD thesis). Oslo School of Architecture and Design.Hsee, C. K., Loewenstein, G. F., Blount, S., & Bazerman, M. H. (1999). Prefer-

ence reversals between joint and separate evaluations of options: a review and

theoretical analysis. Psychological Bulletin, 125(5), 576e590.Jansson, D. G., & Smith, S.M. (1991). Design fixation.Design Studies, 12(1), 3e11.Kahneman, D., & Tversky, A. (1979). Prospect theory: an analysis of decisions

under risk. Econometrica, 47(2), 263e292.

Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psycho-logical Review, 80(4), 237e251.

Kant, E. (1985). Understanding and automating algorithm design. IEEE Transac-

tions on Software Engineering, SE-11(11), 1361e1374.Kihlander, I. (2011). Managing concept decision making in product development

practice. (Doctoral thesis). KTH, Royal Institute of Technology, Department

of Machine Design.Laakso, & Liikkanen. (2012). Dubious role of formal creativity techniques in pro-

fessional design. In Proceedings of the ICDC2012. Glasgow, UK, 18e20

September.Legardeur, J., Boujut, J. F., & Tiger, H. (2010). Lessons learned from an empirical

study of the early design phases of an unfulfilled innovation. Research inDesign, 21(4), 249e262.



org/10.1016/j.destud.2014.02.006









http://www.designcouncil.org.uk/webdav/harmonise?Page/@id.53&Session/@id.D_4jaHtwk0Hj7ve5elIToe&Document/@id.10149
































































26



Liikkanen, L. A., & Perttula, M. (2009). Exploring problem decomposition inconceptual design among novice designers. Design Studies, 30(1), 38e59.

Litwinski, L. (1947). The psychology of “mine”. Philosophy, 22(83).L�opez-Mesa, B., & Bylund, N. (2011). A study of the use of concept selection

methods from inside a company. Research in Engineering Design, 22, 7e27.Mattson, C. A., & Messac, A. (2005). Pareto based concept selection under uncer-

tainty, with visualization. Optimization and Engineering, 6(1), 85e115.Mueller, J. S., Melwani, S., & Goncalo, J. A. (2012). The bias against creativity:

why people desire but reject creative ideas. Psychological Science, 23(1), 13e17.Nesselroade, K. P., Beggan, J. K., & Allison, S. T. (1999). Possession enhance-

ment in an interpersonal context: an extension of the mere ownership effect.Psychology & Marketing, 16(1), 21e34.

Nikander, J. B., Liikkanen, L. A., & Laakso, M. (2013). Naturally emerging de-

cision criteria in product concept evaluation. In Proceedings of ICED’13.Seoul, Korea, 19e22 August.

Nishimura, S., Nevgi, A., & Tella, S. (2008). Communication style and culturalfeatures in high/low context communication cultures: a case study of Finland,

Japan and India. In A. Kallioniemi (Ed.), Renovating and developing subject di-dactics. Proceedings of a subject-didactic symposium in Helsinki on Feb. 2, 2008.Part 2 (pp. 783e796). University of Helsinki, Department of Applied Sciences

of Education, (Research report).Pahl, G., Beitz, W., Feldhusen, J., & Grote, K. H. (2007). Engineering design: A

systematic approach. Berlin: Springer.

Pierce, J. L., Kostova, T., & Dirks, K. (2001). Toward a theory of psychologicalownership in organizations. The Academy of Management Review, 26(2),298e310.

Pierce, J. L., Kostova, T., & Dirks, K. T. (2003). The state of psychologicalownership: integrating and extending a century of research. Review of GeneralPsychology, 7(1), 84e107.

Pugh, S. (1991). Total design: Integrated methods for successful product engineer-

ing. Wokingham: Addison-Wesley Publishers Ltd.Reb, J., & Connolly, T. (2007). Possession, feelings of ownership and the endow-

ment effect. Judgement and Decision Making, 2(2), 107e114.

Rittel, H. W. J., & Webber, M. M. (1984). Planning problems are wicked prob-lems. In N. Cross (Ed.), Developments in design methodology (pp. 135e144).New York, NY: John Wiley & Sons.

Roozenburg, N. F. M. (1977). De basis ontwerpcyclus. Collegedictaat Ontwerpme-thodologie. TUD-IO.

Roozenburg, N. F. M., & Eekels, J. (1995). Product design: Fundamentals andmethods. Chichester: John Wiley & Sons.

Schmidt, J. B., & Calantone, R. J. (2002). Escalation of commitment during newproduct development. Journal of the Academy of Marketing Science, 30(2),103e118.

Sch€on, D. (1983). The reflective practitioner: How professionals think in action.New York: Basic Books.

Shah, A. K., & Oppenheimer, D. M. (2007). Easy does it: the role of fluency in cue

weighting. Judgment and Decision Making, 2(6), 371e379.Shaw, A., Li, V., & Olson, K. R. (2012). Children apply principles of physical

ownership to ideas. Cognitive Science, 36(8), 1383e1403.Stanovich, K. E. (2003). The fundamental computational biases of human cogni-

tion: heuristics that (sometimes) impair decision making and problem solving.In J. E. Davidson, & R. J. Sternberg (Eds.), The psychology of problem solving(pp. 291e342). Cambridge: Cambridge University Press.



.org/10.1016/j.destud.2014.02.006










































































Stanovich, K. E., & West, R. F. (1998). Individual differences in rational thought.Journal of Experimental Psychology, 127(2), 161e168.

Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: impli-cations for the rationality debate? Behavioral and Brain Sciences, 23(5),

645e726.Stevanovi�c, M., Marjanovi�c, D., & �Storga, M. (2012). Decision support system

for idea selection. In International design conference e Design 2012. May

21e24, 2012, Dubrovnik, Croatia.Taylor, S. E., & Brown, J. D. (1994). Positive illusions and well-being revisited:

separating fact from fiction. Psychological Bulletin, 116(1), 21e27.Trickett, S. B., & Trafton, J. G. (2009). A primer on verbal protocol analysis. In

Schmorrow, D, Cohn, J., & Nicholson, D. (Eds.). (2009). The PSI handbook ofvirtual environments for training and education, Vol. 1 (pp. 332e346). Westport:

Praeger Security International.Turk, D. J., van Bussel, K., Walter, G. D., & Macrae, C. N. (2011). Mine and me:

exploring the neural basis of object ownership. Journal of Cognitive Neurosci-ence, 23(11), 3657e3668.

Ullman, D. G., Herling, D., & Sinton, A. (1996). Analysis of protocol data toidentify product information evolution and decision making process. InN. Cross, H. Christiaans, & K. Doorst (Eds.), Analysing design activity (pp.

169e185). Chichester: Wiley.Ulrich, K. T., & Eppinger, S. D. (2003). Product design and development (3rd ed.).

New York: McGraw-Hill.

Vinodh, S., & Girubha, R. J. (2011). Sustainable concept selection usingELECTRE. Clean Technologies and Environmental Policy 1e6.

Wilson, T. D. (1994). Commentary to feature review. The proper protocol:

validity and completeness of verbal reports. Psychological Science, 5,249e252.



org/10.1016/j.destud.2014.02.006










































The preference effect in design concept evaluation

Documents

Transcript of The preference effect in design concept evaluation