Fields for co-creation background for Rhyme

Birgitta Cappelen, AHO Oslo school of Architecture and Design, Institute for Design

IFI/UiO, 18 oktober 2011

My background

Education

Phd Couses in Interaction Design, K3, Malmö University College

MA in Industrial Design, AHO

BA in Humanities from Oslo University (Cand. Mag) Media and Communication Studies (2 years)

Computer Science (div courses)

History of Ideas and Philosophy (2 years)

BA-level in Economy from BI

Experience 20 years experience from industry

Consultancy / Product Development

User Interface Design / Interaction Design

Service Design, ERP, BPR

Brand Building

Multimedia Design, Set Design

Method development

Started 3 companies (New View, Interaction Design, Creuna)

10 years experience from research KTH - Smart Things and Environment

Interactive Institute – Narrativity

K3 – Malmö Högskola

Design Sciences – Lund University

AHO-Design-Rhyme (2010 - 2015)

Artgroup MusicalFieldsForever (2000 – 2010)

Intuition and ambition

Signs of use

Signs of wear

The beauty of traces in physical objects

Inscription of ourselves caused by reuse of the object

Openness

• Openness in the interpretation and doing

• No Right or Wrong

• Reduse dependensies in sequences

• From any point there are many ways

• From objects to Relation

Change the power structure

Starting research question

From What is “good” design?

How does one create “good” design?

To What is “meaningful” design?

How does one design meaningful smart objects?

Sub questions • What does a designer create?

• What is a smart interactive object?

• What do I mean with meaningful as a design quality?

• For whom is it meaningful?

• What do I mean with use?

Suggestions…

What is “good” design?

From “good” design

To meaningful design

• Expresses identity

• Strengthens identity

• Builds relations between the actors over time

• Expand the discourse field

• Shapes the culture

Suggestions

What does it mean to use?

From use

Via consumption and production (prosumer)

To re-creation

• creation of meaning through

mediated communication

• recreation of the mediated articulation in the discourse field,

the network, the context

Suggestions…

What qualities must the design have to be meaningful?

From ”user friendly” and controlled

To action space and negotiation space • Openness

• Ambiguity

• Appropriate able /Adaptive / Transformable

• Circulateable

• Developing over time

• Acknowledgeable

• Offer relevance over time

• Negotiable over time

• Narrative structured

like a field

a field …

A field is something we cultivate where what we plant grows. It is an open

area to walk through. It is an area of knowledge with yet unidentified

constraints.

A field is ambiguous.

Suggestions

User friendly and efficient

in performing functions

Co-creator

Use

User

Good design

Defined functions

Object

Recreation

Potentialities

Field

Multivalent design

Ambiguity, openess, trans-formable, circulateable, inscribeable

Unfoldings - a field

Unfoldings, Stockholm House of Culture, 2003

Unfoldings, Rooseum Malmö, 2003

Unfoldings, ICA, 2003

Other fields...

IFI-BCAKK.ppt

1

MUFI, Malmöfestivalen 2002

2

3

4

5

samtidsmuseet Roskilde 2004

6

Strainings, Neon, 2004

7 Strainings, H05, 2005

8

TigerTales, Hässleholm House of Culture, 2005

9

a new path 2006 -

10

11

background UbiComp and Tangible interaction unique potential for disabled

UbiComp and Tangible interaction challenge universal design

MSE (Snoezelen) are established method

Music has unique communication potentials for this group

No or little work on music therapy and interactive music

AAC mostly pictograms and signs

Exists few possibilities for collaboration

12

what we have done Based on this earlier work (musicalfieldsforever)

Develop interactive, audio-tactile experience environments(fields)

Children with severe disabilities and their families

To facilitate collaboration and communication

between all family members

On equal terms (not in the same manner…)

“Smart” textile active structures

Communication through , with, in music

MUSICKING (C.Small)

13

The Field Vinings, IKDC, 2007

14

traditional situation

- one knows, one does not

15

Our interactive fields Designed and loaded with potentiality

Open to many

∙ interpretations

∙ interaction forms

∙ activity levels

∙ relations

Simultaneously

Develops over time

16

Motivate to musical actions Our musical fields motivate the user to

∙ play as on musical instruments

∙ communicate as with musical actors (co-musicians)

∙ experience as being in a soundscape

17

communicate with musical actors, co-musicians

18

Musical Gestures

iterative

impulsive

sustained

complex

19

testing the accelerometer

20

testing microphone and sound analysis

21

testing speakers as part of the physical experience

22

bend sensor

lighting

speakers

dynamic graphics

microphones

ORFI

23

touch sensor

lighting

speakers

dynamic graphics

motion sensor (accelerometer)

microphone

touch sensor (capacitive)

Vinings

24

Open and Ambiguous

The user can create many interpretations and actions

The Field simultaneously offers many

∙ roles (instrument, actor, soundscape)

∙ interaction forms

∙ activity levels (passive-active)

Direct and communication like response

Many active sensors at the same time

25

the user can create many interpretations

26

open to many forms of interaction and activity levels

27

open to many forms of interaction

28

open to many interpretations and actions

29

talk, sing, wonder, relax, interact, jump, experience, share

30

Changeable

Modular

∙ motivate to use over time

∙ possible to change, distribute and restructure

∙ user can adopt many roles

Transformable

∙ dynamically changeable form, new functions

Inscribable

∙ individual interpretation

∙ personification

∙ physical and auditive tagging/sampling

31

motivating to change, distribute and restructure over time

32

users create new functions through changeable form

33

personificate by physical and auditive inscription (tag)

34

Consistent and Suprising

Consistency between sound and interaction form

∙ small flexing wire curls motivate iterative gestures

∙ large dimensions motivate impulsive gestures

∙ consistent with character of the Field

Registration of presence offers long use sequences

Surprising invitations motivate communication

Unpredictable wire curl encourage communication

35

homogeneous textile supports the background soundscape

36

small flexing wire curls motivate iterative gestures

37

the empty field waits and invites to interaction

38

continuous flexing wire offers rhythmic variation

39

skin like, mat finish, user compares with touching

40

stretch textile with bouncing qualities

41

the ORFI field

42

a father plays with iterative gestures

43

they share ambient experiences

44

they communicate

45

she wonders

46

mother and boy communicate in the Field

47

he hits !

48

they tease

49

he hides

50

the Field answers in music and rhythmic glow

51

they roll and throw

52

he climbs sustained

53

they share

54

they play, communicate and experience

55

Testing february 2009 A therapist at Rosenlund Hospital after the testing perion in

february 2009:

”Rummet ger möjlighet till närhet, kommunikation, att vara

tillsammans, att upptäcka med alla sinnen, att påverka själv,

inget är ”fel” allt är ”rätt”. Samspelet blir naturligt och

självklart, det går inte att undvika. En hjälp till ”närvaro i

stunden”.

Verkligen det vi vill!

56

Experiences Huge potential for this group

More work with sensors/materialisation of sensors

More work on voice (filters, clipping, composing)

More work on spesific genres MINI, VOXX,

More work on individual setups

More work on hardware ( robustity, battery, charging)

Get it on Internet

More work on sosial distributed aspects (social media)

More work with musical therapie and communication strategy

57

next step

-

RHYME

RHYME co-creation through

tangible interaction

and music

VERDIKT

5 years

10.mill NOK

VERDIKT – The future of Internet

Objectives

from application

1 .

“Radically improve the health

and well-being of people with

severe disabilities, through a

new treatment paradigm based

upon collaborative, tangible, net

based, "smart objects", with

multimedia capabilities.”

2 .

“Contribute to the emerging

body of knowledge in interface

design, specifically within the

area of networked social co-

creation, based on cross-medial

interaction through the Internet

of Things.‖

Further objectives are:

Develop knowledge through design of new concepts of

Internet of Things. Things that facilitate play, communication

and co-creation, between people with and without severe

disabilities, on equal terms, in different communicative

situations

Identify the new field and its potential that emerges, by

bringing together Interaction Design, Tangible Interaction,

Universal Design, Music & Health

Establish an international network for the field

Contribute to collaborative courses between the partners

Perspectives...

Ecological and Humanistic Health

approach (Blaxter, 2004, 2010)

Everyday use of music (deNora 2000, Ruud 2006)

Research by design (Schön, 1983) and

User-centred design (UCD)

Art and design perspectives on technology

...perspectives

Technology as instrument, actor and

ambient landscape (Latour 1999, DeNora 2000)

Openness (Eco 1962, Löwgreen 1999/2004)

and Design for diversity

Narrative structured, Dynamic affordance

and Cross-media

3 partners

Norwegian Academy of Music

Center for Music and Health

Even Ruud,

3 x Professor, Musicology, Music therapy, Music and

Health, Founder of Center for Music and Health

Musician, Musicologist, Psychologist, Music therapist,

author and co-author of 40 books

Karette Stensæth,

Ass. Professor in Music therapy

20 years music therapist, Phd in Music therapy on

collaboration and improvisation

document health effect

publish in the music and health and

music therapy community

Institute of Informatics

University of Oslo

Design Group

Jo Herstad,

Ass. Professor in Informatics, Phd in Informaticts

on use of mobile communication,

Specialist in Universal Design and

mobile communication

IFI: HCI and Universal design

perspective

oxo

AHO Oslo School of Architecture and Design

Interaction Design Group

(project owner)

AHO: design and prototype development

research by design methodology

Birgitta Cappelen,

Ass. Professor in Interaction design at AHO,

Industrial designer, Interaction designer, Service

designer

Anders-Petter Andersson, senior lecturer at

Interactive Sound Design, Kristianstad University

Sound designer, Musicologist

Fredrik Olofsson, Composer, Programmer, Video

artist, lecturer at Universität der Künste Berlin

worked together over 10 years, met at ii

musicalfieldsforever.com

RHYME

to rhyme, a rhyme

co-creation with tangible,

spatial, cross-media

by communicative, musical and

narrative principles

to improve health and well-being

for people with servere disabilities

samskapende ting

co-creative tangibles

25 family plays on and with ORFI

Background

music effects health

scientifically documented over the

last 15 years

music therapy

multisensory room- Snoezelen

white multisensory room

the Internet of Things

Ubiquitous Computing

research in interactivity and

narrativity

interactive music composition

38 user tries out the microphone and the VOX music genre in ORFI

co-created digital

narratives and experiences

social media

interactive art

TigerTales, Hässleholm House of Culture, 2005

Searching Voices, World New Music Days, 2009

design for all

for diversity

Goals

increased health through

(Ruud, 2006)

reduction of

isolation and passivity

51 friends playing in ORFI

evoke feelings

53 family plays in ORFI

increase ability to master

create social relations

56 family spending time together in Vinings

create experience of

meaningful wholeness

58 users experience sounds and vibrations in Vinings

with co-creative things,

co-creative tangibles

ICT- based things

tactile interaction

63 Vining’s accelerometers are being tested

music, actions and gestures

Strainings, Neon, 2004

light and dynamic graphics

67 Unfoldings, Rooseum Museum of Contemporary Art, Malmö, 2003

cross-medial, spatial

interaction

User group

children with severe

disabilities

and their families

71

one of Norway’s leading schools and centers within the field

Related things

Soundbeam

SoundBeam in use

Paro

Japanese PARO is bought by Oslo Municipal

The project

4 empirical studies

3 new prototypes

different design qualities

and situations

Situations

to facilitate co-creation between

2 persons simultaneously

to facilitating simultaneous co-creation, in

social networks

tangible cross-media solution to facilitate co-

creation, distributed in time and space

start with existing ORFI,

created earlier by 3 (Birgitta, Anders-Petter, Fredrik)

project participants to

create common foundation

83 Alina in ORFI

bend sensor

LED lights

speakers

dynamic graphics

microphones

Spring 2011

action research, 5 actions

weekly interviews

video observations

Generation 2

hardware explorations

3 new hardware plattforms

iDevice

(iPhone)

RHYME wireless

(Android, Google I/O ADK)

RHYME wired

(Arduino Mega, Shell, MiniMac)

tactile textile touch sensors

ORFI – too hard for some

e-textile explorations

different light , light spreading and textile techniques

creating tangible tactile glowing sensors

cross-media explorations

cross-media interaction with camera and projection

people with dementia remember with music

Just Now

E-textile and musical interaction-course

AHO-KhiO

Presentation 4. nov. 13.00 KhiO

Other possibilities

other target groups: dementia

other places: institutions

other situations: therapy sessions

other places: institutions

other relations: home,

institutions, users, care persons

people with dementia remember with music

other combinations:

co-creative things

social media

service design

great potential,

social and economical

a new paradigm for health

improvement ICT ?

back to the start

So... Forget about tables, chairs, sofas and floors

The new potentialities lies in between, in hybridizations, Work and leisure

Reading and Writing

Tables and Sofas

Designers and users

etc. in intertextuality

My suggestions is to create multivalent fields of potentialities for co-creators instead of user-friendly objects with functionalities for users

Its another way of thinking and designing

Fields for co-creation

My inspirators…

Aristotle (rhethorics, poetics, praxis, phronesis)

Goffmann E. (The presentation on self in everyday life)

de Certau M. (The Practice of Everday Life)

Miller D. (Material Culture)

Appadurai A. (The social life of things)

Barthes R. (Mythologies, The pleasure of the text, other…)

Eco U. (The open works)

Hall S. (Identity theory, Encoding/DeCoding)

Giddens A. (Identity theory, Modernity)

My inspirators…

Bourdieu P. (Distinction, Theory on Power/Praxis, )

Foucault M. (Discourse theory, Archeology/Genealogy, The Order of things, Power theory)

Arent H. (Human Condition)

Nardi B. (Information Ecology, Contextual Design)

Laurel B. (Computer as Theatre)

Suchmann L. (Situated Actions)

Latour B. (We have never been modern, Pandoras hope, Laboratory life)

Löwgren J. (Thoughtful Interaction, ―Handlingsutrymme‖)

www.musicalfieldsforever.com

References

Aanestad, M., 2003. The camera as an Actor: Design-in Use of Telemedicine Infrastructure in Surgery. Computer Supported

Cooperative Work, 12(1), 1-20.

Andersson, A-P., Cappelen, B., 2008. Same But Different, Composing for Interactivity, Proceedings, AudioMostly08, Interactive

Institute, Piteå

Andersson, G., Carlbring, P., Kaldo V., 2006. Kognitiv beteendeterapi via internet - en behandlingsform för framtiden? Psykisk

hälsa, Vol (2006): s. 50 – 58.

Belson, S. & DiGangi, S., 2003. Ubiquitous computing meets universal design. In D. Lassner & C. McNaught (Eds.), Proceedings of

World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003 (pp. 3185-3187). Chesapeake, VA: AACE.

Bjursell, G., & Vahle Westerhäll, L., (eds) 2008. Kulturen och Hälsan. Santérus förlag, Stockholm.

Bjursell, G., 2009. Research plan, Culture and Health, University of Gothenburg,

http://www.koh.org.gu.se/digitalAssets/1289/1289646_RESEARCHPLAN_2.pdf, accessed 15 november 2009

Blaxter, M., Health, 2004. Cambridge, UK : Polity.

Crockford, D., 2003. Lesson One, An introduction to bliss symbolic language, Blissym Language Institute: www.blissym.com,

accessed Nov 15 2009.

Bruscia, K., 1998. Defining music therapy. Gilsum NH: Barcelona Publishers.

Bruscia, K., 2000. The Nature of Meaning in Music Therapy. (Bruscia interviewed by Brynjulf Stige) Nordic Journal of Music

Therapy, 9 (2).

Cappelen, B., & Andersson, A-P., 2003. From Designing Objects to Designing Fields - From Control to Freedom. Digital

Creativity14(2): 74-90.

Cappelen, B., & Andersson, A-P., 2008. To challenge textile with music. Proceedings, Ambience08, Smart Textile Conference,

University College of Borås, Sweden.

Chang A., Ishii, H. 2006. Sensorial Interfaces, Proceedings, DIS 2006, University Park, Pennsylvania, USA.

CEC, Commission of the European Communities, 1998. The Disappearing Computer, FP5 (1998-2002)

http://cordis.europa.eu/ist/fet/dc-ob.htm, accessed, Nov 15 2009.

CEC, Commission of the European Communities, 2009. Internet of Things — An action plan for Europe, COM(2009) 278, Brussels:

http://ec.europa.eu/information_society/policy/rfid/documents/commiot2009.pdf, accessed, Nov 15 2009.

Crawford, C., 2003. on Game Design, Peachpit.

DeNora, T., 2000. Music in everyday life, Cambridge Univ. Press.

Dourish, D., 2001. Where the Action Is: The Foundations of Embodied Interaction. Cambridge MA: MIT Press.

Dunne , A., & Raby F., 2001. Design noir: the secret life of electronic objects . UK

Gaver, W., Sengers, P., Kerridge, T., Kaye, J., and Bowers, J., 2007. Enhancing ubiquitous computing with user interpretation:

Field testing the Home Health Horoscope. Proc. CHI’07.

Gislén, Yet al., 2008. Avatopia: a cross-media community for societal action, Personal and Ubiquitous Computing, 12, 4, 289-297,

Springer, London.

Greenfield, A., 2006. Everyware. Berkeley, Calif. : New Riders. Greenfield, A., 2006. Everyware. Berkeley, Calif. : New Riders.

Ishii, H., Ullmer, B., 1997. Tangible Bits: Towards Seamless Interfaces Between People, Bits, and Atoms, Proceedings of the

Conference on Human Factors in Computing Systems (CHI '97).

Jenkins, H., 2006 . Convergence culture: Where old and new media collide. New York Univ.Press.

Latour, B., 1999. Pandora's Hope. Essays on the Reality of Science Studies, Harvard University Press.

Laurel, B., 2003. Design Research: Methods and Perspectives. MIT Press, Cambridge, MA.

Latour, B., 2005. Reassembling the social: an introduction to Actor-network theory, Oxford: University Press.

Löwgren, J., Stolterman, E., 2004. Thoughtful interaction design, Cambridge, Mass.: mit Press.

Löwgren, J., 2007. Pliability as an experiential quality: Exploring the aesthetics of interaction design, Artifact, Vol. 1, No. 2. pp.

85-95.

Machover T., 2008. Tod Machover and Dan Ellsey play new music, TED talks:

http://www.ted.com/talks/tod_machover_and_dan_ellsey_play_new_music.html, filmed 2008, accessed Nov 15 2009.

Machover, T., 2004. Toy Symphony, http://www.toysymphony.org/, accessed Nov 15 2009.

Min. of Government Administration and Reform, 2006. An information society for all, Report No. 17 (2006 - 2007) to the Storting:

http://www.regjeringen.no/upload/FAD/Vedlegg/IKT-politikk/stm17_2006-2007_eng.pdf, accessed, Nov 15 2009.

Min. of education and research, 2006. Rammeplan for barnehagens innhold og oppgaver:

http://www.regjeringen.no/upload/kilde/kd/reg/2006/0001/ddd/pdfv/282023-rammeplanen.pdf, accessed, Nov 15 2009.

Min. of education and research, 2008. Hefte om barn med nedsett funksjonsevne:

http://www.regjeringen.no/nb/dep/kd/dok/tidsskrift_nyhetsbrev/forsiden-kd-aktuelt2/kd-ktuelt-nr-3-2008/barnehage/nytt-

temahefte-om-barn-med-nedsett-funks.html?id=506843, accessed, Nov 15 2009.

National Committees for Research Ethics, 2006. Guidelines for research ethics in the social sciences, law and the humanities:

http://www.etikkom.no/en/In-English/Publications, accessed, Nov 10 2009.

National Committees for Research Ethics, 2008, Guidelines for research ethics in science and technology:

http://www.etikkom.no/en/In-English/Publications, accessed, Nov 10 2009.

Pagliano, P. J., 1999. Multisensory environments. London: David Fulton Publishers.

Redström, J., 2008. Tangled Interaction: On the Expressiveness of Tangible User Interfaces. ACM ToCHI (Transactions on Computer-

Human Interaction), Vol. 15, No. 4 (Special Issue on the Aesthetics of Interaction), pp. 1-17.

Ruud, E., 1990. Musikk som kommunikasjon og samhandling.Teoretiske perspektiv på musikkterapien: Solum.

Ruud, E., 1998. Music therapy: improvisation, communication, and culture. Gilsum, NH: Barcelona Publishers.

Ruud, E., 2005. Lydlandskap, om bruk og misbruk av musikk, Fagbokforlaget.

Ruud, E., 2006. Musikk gir helse. Musikk og helse. Aasgaard, T., (ed.), Cappelen Akademisk Forlag, Oslo.

Norwegian directorate of Health, 2008. Nasjonale strategi for elektronisk samhandling i helse- og omsorgssektoren 2008 – 2013 –

Samspill 2.0: http://www.helsedirektoratet.no/vp/multimedia/archive/00047/Samspill_2_0_-strate_47719a.pdf, accessed, Nov 15

2009.

Norwegian directorate of Health, 2004. Smarthusteknologi - planlegging og drift i kommunale tjenester:

http://www.helsedirektoratet.no/vp/multimedia/archive/00001/IS-1216_1681a.pdf, accessed, Nov 10 2009.

Schön, D.A., 1983. The Reflective Practitioner: How professionals think in action. Temple Smith, London.

Stensæth, K., 2007. Musical Answerability, A Theory on the Relationship between Music Therapy Improvisation and the

Phenomenon of Action, PhD Dissertation, Norwegian Academy of Music.

Suchman, L., 2007. Human-Machine Reconfigurations: Plans and Situated Actions (2nd edition), Cambridge: Cambridge University

Press

Tetzchner, S., & Martinsen, H., 2002. Alternativ og supplerende kommunikasjon. Oslo: Gyldendal.

Weiser, M., 1994. The world is not a desktop. interactions, 1, 1, 7-8.

Weiser, M., 1991. The Computer for the 21st Century, Scientific American, September.Stensæth, K., 2007. Musical Answerability, A

Theory on the Relationship between Music Therapy Improvisation and the Phenomenon of Action, PhD Dissertation, Norwegian

Academy of Music.

TAKK !

RHYME.no

Birgitta Cappelen birgitta.cappelen@aho.no