Users as a Hidden Resource for Creativity: Findings From an Experimental Study on User Involvement

Users as a Hidden Resource forCreativity: Findings from anExperimental Study on UserInvolvement

Per Kristensson, Peter R. Magnusson andJonas Matthing

The main objective of this article is to report the empirical findings from a study on userinvolvement in service innovation. In doing this, we seek to answer the question of how userinvolvement affects the originality of new service ideas. An experimental investigation wascarried out which included 54 participants arranged into three groups of creative users,ordinary users and professional service developers. The empirical data revealed that the usersproduced more original ideas than the company’s professional service developers. It is thussuggested that business organizations attempt to innovate original products would benefitfrom involving their customers.

Introduction

A ll innovation begins with creative ideas(Amabile, 1996; Amabile, Conti, Coon,

Lazenby & Herron, 1996). The developmentof successful services, the implementation ofnew processes, the design of new productsand their introduction onto the market alldepend on a person or a team coming up witha good idea and developing this idea beyondits initial state. Launching a novel product,based on an original idea, in the field willincrease the chances of gaining market share,thus implying a major financial advantage fora company. One critical phase in product andservice development is the early idea phase.Nevertheless, one of the least understoodaspects of innovation is real leaps of creativity(Cooper, 1993).

Research concerning products that areperceived as being creative reveals that theyelicit a distinct set of aesthetic responses fromobservers, e.g. surprise, satisfaction, stimu-lation and savouring (Amabile, 1996). Earlyfindings (Guilford, 1950; Barron, 1955) con-cluded that originality was an important di-mension of a creative new product. Creativityresults in the production of some novel out-

put that is satisfying and represents a realleap forward from the current state of the art(Stein, 1974). The originality of a product isexplained by its uncommonness in a particu-lar situation and its applicability to a givenproblem (Amabile, 1996). Recent research hasidentified some explicit product character-istics (dimensions), as discriminating signs ofa creative product (Amabile, 1996; Besemer &O’Quin, 1987). The employment of dimen-sions in order to assess creative products isultimately considered as the most usefulprocedure for creativity research in general(Amabile, 1996). According to Besemer andO’Quin (1987), dimensions that capture anew product, product ideas, or creativity ingeneral, are characterized by novelty, resolu-tion, and elaboration. The degree of origin-ality is implied by the dimension novelty. Inthe literature (e.g., Isaksen, 1987) novelty iscommonly and frequently referred to as themost obvious attribute of creativity in pro-ducts. The extensive interest in the noveltydimension is perhaps explained by the factthat uniqueness of ideas is being held as animportant criterion for product success (Booz,Allen & Hamilton, 1982). In other words, thefuture of a company today, is to a great extent

Volume 11 Number 1 March 2002# Blackwell Publishers Ltd 2002. 108 Cowley Road, Oxford OX4 1JFand 350 Main St, Malden, MA 02148, USA.

A HIDDEN RESOURCE FOR CREATIVITY 55

determined by the potential of their productportfolio under development. Hence, oneimportant objective for an organization is tohave the capability to present unique, andthus innovative, products. As many companieshave the ambition to be innovative, originalityis the concept that enfolds the innovativedimension.

Today, both scholars and practitioners haveplaced an emphasis on unique ideas (Cooper,1993). This uniqueness, note, applies to theuniqueness perceived by the customer (Craw-ford, 1977; Stevens, Burley & Divine, 1999).On the basis of these findings, normativeresearch has emphasized the involvement ofcustomers in the development of new prod-ucts and services. Since the customer usinga new product or service always ends up asthe adjudicator of this product, and therebyits success, the research literature has pro-posed customer involvement in new productand service innovation (e.g., Prahalad &Ramaswamy, 2000). More specifically, thecustomer is thought of as a co-producer andidea generator for new products and services(Prahalad & Ramaswamy, 2000; von Hippel,Thomke & Sonnack, 1999; Ramirez, 1999;Wikstrom, 1995). The logic behind thisreasoning is that if the customers are the oneswho can decide whether a product idea isunique or not, then he or she should bethought of as a valuable source for initiatingprofitable ideas. This greater focus on thecustomer is not, perhaps, entirely new. Theconception of the customer as a co-producer,however, is another step forward. Accordingto Wikstrom (ibid) deepening the interactionbetween customers and manufacturers im-proves the level of creativity. Since creativityplays an important role in the front-endinnovation phase, co-opting customer compe-tence, and involving them into the process,ought to be extra contributing in productdevelopment projects.

However, whether or not customers or usershave really contributed with more creativeideas has not been thoroughly investigated inprevious research. There are indications thatusers are the real source of many innovations(von Hippel, 1988), but this does not answerthe question whether or not users contributewith more creative product or service ideasthan the company itself.

Purpose and scope

The purpose of this paper is to report theempirical findings from a study on userinvolvement in service innovation. The focusof the paper is on the users’ contributionsto the originality of the generated ideas.

Thus, the research question is formulated asfollows: How does user involvement in serviceinnovation affect the originality of the service ideasproduced?

Method

A working hypothesis for the study has beenthat user involvement in the service innova-tion process does affect the end-result withregard to unique ideas. In other words, it hasbeen assumed that user involvement willmake a difference compared to the involve-ment of only professional service developersin the development process – without theinteraction of users. The interest is to pin-point the actual differences and learn how toutilize them for improving user involvement.A cornerstone of our research has been tosimulate a real situation of user involvementas realistically as possible. In other words,the research design itself should constitute arealistic way of organizing user involvement.The context chosen was the development ofend-user services based on the GSM (GlobalSystem for Mobile Telecommunications) stand-ard SMS. These services can be categorized astechnically based self-services (Dabholkar,1996). The platform used in the study iscalled Unified Services (US). US is essentiallya converter between SMS1 messages (textmessages via the mobile phone) in GSM andhttp-calls on the Internet. From the users’point of view, US enables access to informa-tion on the Internet by sending and receivingSMS-messages. Furthermore, US can also beused for remote control. It is for examplepossible to create a service that can switchlights on and off in a home or a radiator in abuilding, or check whether a door is lockedby just sending an SMS.

Sample

All non-professional participants, i.e. theusers, were volunteered students from Karl-stad University, recruited during lectures.The reason for choosing students is that theyrepresent one of the most frequent usergroups of the GSM/SMS service in focusduring the study and, thus making them,realistic users. The professional participants inthe control group worked as service devel-opers at the R&D department of a leadingtelecommunications company in Sweden.

Design

In the CuDIT (Customer Driven IT) project anexperimental method was chosen. The basic

CREATIVITY AND INNOVATION MANAGEMENT56

Volume 11 Number 1 March 2002 # Blackwell Publishers Ltd 2002

idea was to design a study that comparednew services generated by professional servicedevelopers at a company to services gener-ated by users. In doing so, conclusions couldbe drawn regarding the actual value of userinvolvement by way of their contribution inan experimental setting. The design was aMultiple-Group Posttest Design (Spector,1993, p. 38) using three groups. The first wasa control group consisting of 12 professionals inservice development, i.e. this group did nothave any user involvement. The other twogroups consisted of 19 and 17 ordinary users,represented by students in non-technicalstudy programmes, e.g. social science, teachertraining, business administration, etc. Thedifference between the two user groups wasthat in group three (with 17 participants) thestudents had been educated in differentcreativity techniques (henceforth creativeusers). The independent variable was the userinvolvement strategy (type of user) and thedependent variable was the originality ofservice idea (see Table 1 for design).

Limitations

From a methodological perspective, the de-sign has two weaknesses. It does not use trulyrandom assignment of the participants, nordoes it have full control of the process duringthe experiment. Random assignment wasnot practically possible for any of the user-groups. Group one was picked at random;however the two other groups were volun-teers. Although groups two and three werenot assigned at random, we claim that theyare most probably representative of the groupof people that would participate in this kindof endeavor. The argument for this is that thewhole setting of the experiment was designedin a similar manner to what could be em-ployed in a real situation. The other problemto tackle is the limited amount of control overthe experimental process. During the ideageneration (12 days), the participants workedwith the task on their own, though with thepossibility of contacting us if problems arose.

To deal with this, all participants wereequipped with a logbook (diary). After com-pletion of the trial, all participants wereinterviewed as well; particularly about theincidents that took place around the ideasthey had reported. Process data was thuscollected from both the diary and the inter-views. However, this data is not included inthe analysis in this paper.

Procedure

The actual experiment consisted of fourstages; initiation, idea generation, termination,and evaluation (see Figure 1).

Initiation

At the initiation meeting, users were gatheredand the scope of the study was outlined. Thenthe application platform, US, was demon-strated to the participants. The purpose wasto give them a feeling of the range ofpossibilities for these kinds of services. Thetask and instructions were handed out, inboth oral and written format. All participants,with the exception of the professional servicedevelopers, were given the task of creatingservice ideas that they perceived as valuableto themselves. The experts, on the other hand,were instructed to design services that theythought would bring added value to thestudents at Karlstad University. By carryingout these different formulations, all groupswere actually trying to satisfy the same targetgroup, namely the students at KarlstadUniversity. Consequently, it was possible tocompare the ideas created for new services.

The participants were instructed to docu-ment the idea creation process in a diary thatwas handed out to them. The purpose of thediary was twofold, to function as a methodfor triangulation and to collect process data.

The participating users were also equippedwith mobile phones containing a specialaccount, since they were to come up withservices ideas for this type of equipment. Toprovide the participants an even better senseof how these services work, they were given

Table 1. The Design of the Experiment

Independent variable Process Dependent variable

Type of user:– Professional developers– Ordinary users– Creative users

Idea generation of new services formobile telephony

Creativity:– Originality


# Blackwell Publishers Ltd 2002 Volume 11 Number 1 March 2002

access to a sample of 10 already implementedservices. To be able to test these, and to gainmotivation during the experiment, the phoneshad a credit of approximately 252 and a so-called chat-board. All participants receivedhands-on training on how to use the phonethrough testing some of the services.

Idea generation

The generation of ideas lasted for 12 days.During this period, participants were to createservice ideas by themselves and log them intheir diary. An estimate of the amount of timespent on idea generation was on averageestablished to half an hour each day.

Termination

After 12 days, the idea generation period wasconcluded and a meeting was held. All equip-ment was returned and the participants sub-mitted their ideas into a service description.These were written descriptions of the serviceideas generated in a pre-defined format.

Evaluation

After all trials had been concluded and theservice descriptions were collected, the eva-luation phase followed. In the CuDIT projectseveral dimensions of the service ideas havebeen evaluated, here we will discuss one ofthem, originality. In order to evaluate them,we used one panel consisting of expertswithin the given domain (Amabile, 1996),mobile services, and one panel of usersconsisting of university students (see belowfor a more detailed description).

Measures

The assessment method was based on theConsensual Assessment Technique (CAT),(Amabile, 1996). CAT uses a panel of inde-pendent judges to evaluate different designspresented to them. ‘Design’ is a generic wordfor the object to be evaluated. The techniquebuilds upon comparing different designs andjudging them in relation to each other, as wellas ranking them.

The type of judge evaluating a certain resultcan affect the evaluation, and thus also the(research) question of the value of involvingusers. For example, a company expert mightevaluate the perceived user-value very differ-ently to an ordinary user. In the CuDIT project,two different panels of judges were used toevaluate the ideas. These were (Panel 1) experts(professionals) and (Panel 2) customers (here interms of students from Karlstad University),both groups familiar with mobile telephonyservices.

Before evaluating the originality dimen-sion, the rating system was described bothorally and in writing. A scale of one to tenwas used for rating the originality of theservice ideas generated. In order to ‘calibrate’the judge’s perception of the dimension to beevaluated, a test round was conducted. Fiveservice descriptions were picked and assessedindividually by the judges. After the indi-vidual assessments, the results were discussedin the group. The purpose of this round wasto ensure that the judges had understood howthe dimension was to be evaluated.

After this, the judges rated all the serviceideas (objects) for the dimension in question.

Figure 1. Outline of One Trial in the CuDIT Experiment



The assessments were made individually.Each judge had a paper copy of each servicedescription.

Results

Interjudge reliabilities

Ahead of analyzing the research question, aninterjudge reliability test between Expert andCustomer panels was carried out. A Pearsonbivariate correlation test showed significantresults for the Expert panel as regards theirassessments of the originality variable. Thestrongest correlation (Pearson’s r) obtained was0.62 ( p < 0.001) and the weakest 0.29 ( p = 0.009).The judges’ evaluations of the originality of thegenerated ideas was perceived as satisfactory.Therefore, the individual scores were averagedto a mean score for further statistical analysis.Similar to the Expert panel, an interjudgereliability test was carried out for the Customerpanel. Overall, the Customer panel showedsignificant correlation as well, although onepair of ratings turned out non significant. Thestrongest correlation (Pearson’s r) receivedwere 0.57 ( p < 0.001) and the weakest 0.05( p = 0.626). Although two judges showed anunsatisfactory agreement the overall scoringwas estimated to be appropriate for furtherstatistical analysis. Consequently, the indivi-dual scores were averaged to a mean score.

The effect of user involvement on originality

The dependent variable, the originality of thegenerated service ideas, was tested by meansof a one-way ANOVA. Table 2 shows thescoring for the various independent variables,according to the two panels.

In respect of panel one (Experts) a one-wayANOVA showed significant differences re-

garding the originality between the groups[F (2, 242) = 7.33, p < 0.001]. A post hoc com-parison (Scheffe) indicated that the creativetrained users had more original service ideas ascompared to the company experts ( p = 0.011).The difference between the company expertsand ordinary users was close, but not, sig-nificant ( p = 0.102). In respect of panel two(Customer) a one-way ANOVA also showedsignificant differences regarding the origin-ality between the groups [F (2, 242) = 4.87,p = 0.008]. A post hoc comparison revealed thatthe creative users had more original serviceideas as compared to the company experts( p < 0.001). There were no significant differ-ence between the company experts and theordinary users ( p = 0.413).

Discussion

The present study challenges the normativeview that organizations need to consider thevoice of the customer when developing newinnovative products. This was done by em-pirically investigating company professionalsand users involved in the idea generation ofservice ideas. The assumption that customerinvolvement in the early stages of productdevelopment would result in more originalideas is supported by the present findings.There are primarily two interesting resultswhich can be derived from the study andwhich will be discussed here:

1. Customers generate ideas that are moreoriginal than the ones generated by thecompany.

2. Customers generally assess innovativeideas different from the company.

It is an interesting fact that, in any event,the creativity trained users generate more

Table 2. Mean scores (and SD) for the originality across the three groups

Panels Groups n M SD

Panel one 1 Company experts 55 2.99 1.07(expert judges) 2 Ordinary users 123 3.55 1.79

3 Creative users 67 4.10 1.58Total 245 3.57 1.64

Panel two 1 Company experts 55 4.71 1.38(customer judges) 2 Ordinary users 123 5.05 1.59

3 Creative users 67 5.58 1.71Total 245 5.12 1.60



original ideas than the professional servicedevelopers. One plausible explanation for theresults is that users have different cognitivestyles of solving problems, often referred toas divergent thinking (Guilford, 1967). Con-vergent thinking, which is characterized bylogical reasoning and problem-solving alongestablished principles, is stimulated andsought for in many working situations.Accordingly, professional service developersmay have seen technology grow from aninitial state to a more mature and advancedplatform over the years. Simultaneous to this,creating a very deep understanding of thetechnology itself might also become a burdenagainst creativity. The knowledge creates arigidity in thinking style, professional devel-opers do not think outside the currentcapabilities of the technology. Since thecustomers do not possess the same technicalskills, they are sometimes able to generateideas that integrate, in a novel fashion, tech-nology with their personal environments (i.e.needs and requirements). Furthermore, it ismostly a fact that professional developers andR&D staff do not live in the same environ-ment as their customers, which is why itseems natural that a customer is able toprovide more original service ideas. Afterall, it is difficult to envisage other people’s(i.e., a customers’) situations and unarticu-lated needs and requirements.

In respect of evaluating the ideas, expertsand users made the same group-wise rankingof the originality of the generated service ideas.Overall, however, the expert panel gave theideas a lower score than the customer panel.The reason for this is probably due to thecircumstance that they have more knowledgeof the current state of the art. Accordingly,non professionals seem to be unnecessary asjudges of the relative originality of a group ofservice ideas. On the other hand, the cus-tomer panel appear to perceive the absoluteoriginality on a higher level than the experts,which is why the user is a valuable instru-ment in indicating the absolute originality ofan idea.

The research undertaken can be viewed inthe light of the classic market debate concern-ing the two strategies, or business philosophies,of technology push and market demand. Ininnovation research, recent findings haveproposed a market demand approach, aimingto develop competitive new products by in-corporating and listening to the voice of thecustomer (Griffin & Hauser, 1993). However,most of this research suggests the involve-ment of customers for the sake of customiza-tion, and not for the sake of finding originalproducts (Christensen, 1997). As the empirical

findings of this study suggest, users are ableto create original and unique ideas, incliningtoward the business philosophy of marketdemand. However, before drawing any finalconclusions there is a need to reflect on atleast two additional aspects. The first con-cerns how the originality, or other dimensionsof the service ideas, will be affected by aninteractive exchange of ideas between usersand professionals. In the present study, usersand professionals did not interact, and furtherresearch is needed in order to explore thismatter. The other aspect concerns the ques-tion of whether the service ideas developedare possible to implement from a produci-bility point of view. The ideas generatedproved to be original, but how do these ideasrate in respect of other dimensions, such asusefulness or, as mentioned, producibility.

Taken together, our study indicates thatuser involvement in service innovation cancontribute to the creativity in the service ideasproduced. In respect of the development ofnew services then, one managerial impli-cation of this study suggests that businessorganizations, attempting to produce innova-tive and successful products, have a hiddenresource in their customers.

Notes

1. SMS is an acronym for short-message-service,and is a technology for sending and receivingtext-messages to the mobile phone. SMS isdefined within the GSM specification. GSM isa pan-European standard for mobile telephony.The system was introduced in Europe in 1992,and is today spread all over the world.

2. The cost for sending an SMS message was 0.17or $ 0.15.

References

Amabile, T.M. (1996) Creativity in Context. West-view Press, Boulder, Colorado.

Amabile, T.M., Conti, R., Coon, H., Lazenby, J. andHerron, M. (1996) Assessing the work environ-ment for creativity. Academy of ManagementJournal, 39, 5, 1154–1184.

Barron, F. (1955) The disposition toward origin-ality. Journal of Abnormal and Social Psychology, 51,478–485.

Besemer, S.P. and O’Quin, K. (1987) Creativeproduct analysis: Testing a model by developingjudging instruments. In Isaksen, S.G. (ed.),Frontiers of Creativity Research. Bearly Limited,Buffalo.

Booz, Allen & Hamilton. (1982) New ProductsManagement for the 1980s. Booz, Allen & Hamil-ton, New York.



Christensen, C.M. (1997) The Innovator’s Dilemma.When New Technologies Cause Great Firms to Fail.Harvard Business School Press, Boston.

Cooper, R.G. (1993) Winning at new products: Acceler-ating the process from idea to launch. Perseus Books.

Crawford, C.M. (1977) Marketing research and thenew product failure rate. Journal of Marketing,April, 51–61.

Dabholkar, P.A. (1996) Consumer evaluations ofnew technology-based self-service options: Aninvestigation of alternative models of servicequality. Research in Marketing, 13, 29–51.

Griffin, A. and Hauser, J.R. (1993) The voice of thecustomer. Marketing Science, 12, 1, 1–27.

Guilford, J.P. (1950) Creativity. American Psychol-ogist, 5, 444–454.

Guilford, J.P. (1967) The Nature of Human Intelli-gence. McGraw-Hill, New York.

Isaksen, S.G. (1987) Frontiers of Creativity Research.Bearly Limited, Buffalo.

Prahalad, C.K. and Ramaswamy, V. (2000) Co-opting Customer Competence. Harvard BusinessReview, January–February, 79–87.

Ramirez, R. (1999) Value co-production: Intellec-tual origins and implication for practice andresearch. Strategic Management Journal, 20, 49–65.

Spector, P.E. (1993) Research Designs. In Lewis-Beck,M.S. (ed.), Experimental Design & Methods. Sage.

Stein, M.I. (1974) Stimulating Creativity. AcademicPress, New York.

Stevens, G., Burley, J. and Divine, R. (1999)Creativity + business discipline = Higher profitsfaster from new product developent. Journal ofProduct Innovation Management, 16, 455–468.

Wikstrom, S. (1995) The customer as a co-producer.European Journal of Marketing, 30, 4, 6–19.

von Hippel, E. (1988) The Sources of Innovation.Oxford University Press, New York.

von Hippel, E., Thomke, S. and Sonnack, M. (1999)Creating breakthroughs at 3M. Harvard BusinessReview, September–October, 3–9.

Per Kristensson is Philosophy Licentiateat the Service Research Center, KarlstadUniversity, Sweden. Peter Magnusson, isan Executive PhD Candidate at the Fenixprogram at Stockholm School of Economicsand Telia Mobile. Jonas Matthing is a PhDCandidate at the Service Research Center,Karlstad University, Sweden.



Users as a Hidden Resource for Creativity: Findings From an Experimental Study on User Involvement

Documents

Transcript of Users as a Hidden Resource for Creativity: Findings From an Experimental Study on User Involvement