EMBODIED INTUITIVE INTERACTION IN CHILDREN · products with embedded electronics (referred to as...

EMBODIED INTUITIVE INTERACTION IN

CHILDREN

Shital Desai

B.E (Hons), M.S

Submitted in fulfilment of the requirements for the degree of

Doctor of Philosophy

School of Design, Faculty of Creative Industries

Queensland University of Technology

2017

Embodied intuitive interaction in children i

Keywords

Children

Directly Manipulated Interfaces (DMI)

Embodiment

Embodied Interaction

Embodied Intuitive Interaction

Intuitive Interaction

Interaction model

Physical products

Tangible Embodied Embedded Interfaces (TEIs)

Tactile Interactions

Virtual Interfaces

ii Embodied intuitive interaction in children

Abstract

Products and interfaces for children have continued to evolve in terms of

complexity—moving from traditional physical products to virtual interfaces and

products with embedded electronics (referred to as Tangible Embodied Embedded

Interfaces, or TEIs). As the result of this evolution, there is an increasing need to

design children’s products through the lens of human factors and child-centred

design.

Physical products, virtual interfaces, and TEIs can be placed on a physical-

virtual continuum, and can be interacted with through various interaction modalities.

While Embodied interactions and intuitive interactions with products can result in

positive experiences for children, there is limited empirical research to investigate

the potential for product design to facilitate children’s Embodied intuitive

interaction. To address this research gap, this study focussed on tactile interaction as

an interaction modality to investigate children’s Embodied intuitive interaction.

Two experiments were carried out, with children playing with three types of

toys: physical, virtual, and TEI. Experiment 1 compared a physical toy with an

equivalent virtual app for intuitive interaction and aspects of Embodiment that

facilitate intuitive interaction. Experiment 2 investigated a physical product and a

TEI for intuitive interaction, and the impact of aspects of Embodiment on this

intuitive interaction.

A methodology that enabled a thorough elicitation of aspects of Embodiment

that facilitate children’s intuitive interaction was developed. Children from 5–12

years of age were observed playing with physical, virtual, and TEI products. In

Experiment 1, observations were followed with retrospective interviews. A co-

discovery method was used during observations and retrospective interviews. Data

was analysed using both qualitative thematic analysis and quantitative analysis.

The results suggest that physical products are more intuitive than virtual

interfaces. They also suggest that TEIs can also be intuitive, depending on the

configuration and integration of the system’s physical and virtual elements. Physical

affordances is the primary contributor to intuitive interaction in physical products

Embodied intuitive interaction in children iii

and TEIs, while virtual interfaces rely on perceived affordances for intuitive

interaction. People use natural and deliberate clues to detect physical and perceived

affordances, respectively. In the absence of such clues, affordances can remain

hidden. Symbolic, deliberate clues are often difficult for children to understand, and

they could be misinterpreted (as in the virtual interfaces). This study suggests the use

of Embodied representations as deliberate clues. Emergence, scaffolding, and co-

operative activity were the next most important contributors to intuitive interaction in

physical products and TEIs, and were found to be strongly correlated to physical

affordances.

The results are transferrable to an Enhanced Framework of Intuitive Interaction

(EFII), and provide an enhanced version of the concept. This enhanced version, in

turn, leads to new directions for research in the area. More specifically, the outcome

of this study is the model for Embodied intuitive interaction (MEII) that represents

children’s interaction with interfaces, using tactile interactions. MEII can be used to

design and evaluate children’s interfaces for Embodied intuitive interactions. MEII is

innovative in its evaluation of children’s interactions for Embodied intuitive

interaction. Furthermore, it could be used to design Embodied intuitive products for

children.

The significance of this study is its empirical verification of claims (in the

research literature) that physical products are more intuitive than virtual interfaces.

The study has conceptualised TEIs using a physical-virtual continuum and

interaction modalities. In so doing, it has paved the way for future research on

various configurations that TEIs could offer in terms of physical and virtual

integration. Furthermore, the child-centric and design-centric approach to

Embodiment and intuitive interaction taken in this study, means that Embodiment

need not be visualised as a concept involving bodily movements only, but could also

be conceptualised in terms of design aspects.

iv Embodied intuitive interaction in children

Table of Contents

Keywords .................................................................................................................................. i

Abstract .................................................................................................................................... ii

Table of Contents .................................................................................................................... iv

List of Figures ........................................................................................................................ vii

List of Tables ........................................................................................................................... ix

Statement of Original Authorship ............................................................................................ x

Acknowledgements ................................................................................................................. xi

Chapter 1: Introduction .................................................................................... 13

1.1 Definitions of concepts and terms ................................................................................ 13

1.2 Physical-Virtual continuum ......................................................................................... 17

1.3 Interaction modalities ................................................................................................... 17

1.4 Research Problem ........................................................................................................ 20

1.5 Aims and objectives ..................................................................................................... 23

1.6 Research significance ................................................................................................... 23

1.7 Thesis overview ........................................................................................................... 27

1.8 Summary ...................................................................................................................... 27

Chapter 2: Embodiment and Intuitive Interaction ......................................... 28

2.1 Embodiment ................................................................................................................. 28

2.2 Embodiment in Interaction Design .............................................................................. 36

2.3 Intuition and Intuitive Interaction ................................................................................ 39

2.4 Embodied Intuitive interaction ..................................................................................... 51

2.5 Summary ...................................................................................................................... 52

Chapter 3: Children’s Embodied Intuitive Interaction .................................. 55

3.1 Embodiment in children ............................................................................................... 55

3.2 Directly manipulated interfaces for children ................................................................ 60

3.3 Intuition in children ...................................................................................................... 62

3.4 Children’s Embodied intuitive interaction ................................................................... 64

3.5 Summary ...................................................................................................................... 67

Chapter 4: Aspects of Embodiment .................................................................. 69

4.1 Cognitive aspects of Embodiment ............................................................................... 69

4.2 Design aspects of embodiment..................................................................................... 75

4.3 Relationship between cognitive and design aspects of embodiment ........................... 81

4.4 Summary ...................................................................................................................... 83

Chapter 5: Research Design .............................................................................. 85

Embodied intuitive interaction in children v

5.1 Methodology ................................................................................................................. 85

5.2 Participants ................................................................................................................... 91

5.3 Data collection Methods ............................................................................................... 94

5.4 Analysis ........................................................................................................................ 98

5.5 Summary ..................................................................................................................... 107

Chapter 6: Embodied Intuitive Interaction: Physical and Virtual ............. 109

6.1 Methodology ............................................................................................................... 109

6.2 Analysis ...................................................................................................................... 114

6.3 Discussion ................................................................................................................... 126

6.4 Summary ..................................................................................................................... 129

Chapter 7: Primary Predictors of Embodied Intuitive Interaction ............ 131

7.1 Methodology ............................................................................................................... 132

7.2 Analysis ...................................................................................................................... 136

7.3 Discussion ................................................................................................................... 150

7.4 Summary ..................................................................................................................... 155

Chapter 8: Discussion ...................................................................................... 157

8.1 Design Implications .................................................................................................... 157

8.2 Model for Embodied intuitive interaction (MEII) for children .................................. 164

8.3 Summary ..................................................................................................................... 172

Chapter 9: Contributions and Future Work ................................................. 172

9.1 Contributions to knowledge ........................................................................................ 173

9.2 Research outcomes ..................................................................................................... 176

9.3 Research limitations ................................................................................................... 177

9.4 Future research ........................................................................................................... 179

9.5 Conclusions ................................................................................................................ 180

Bibliography ........................................................................................................... 183

Appendices .............................................................................................................. 207

Appendix A Strength of agreement based on ICC values ..................................................... 207

Appendix B Parametric and non-parametric statistical analysis ........................................... 208

Appendix C Thresholds for Effect size ................................................................................. 209

Appendix D Coding heuristics for Types of Interaction ....................................................... 210

Appendix E Coding heuristics for Aspects of Embodiment ................................................. 211

Appendix F Consent form for children and parents .............................................................. 212

Appendix G Consent form for school principal .................................................................... 214

Appendix H Image release consent forms ............................................................................ 215

Appendix I Image release information sheet itle .................................................................. 217

Appendix J Participant information sheet for experiment 1 at school .................................. 219

vi Embodied intuitive interaction in children

Appendix K Participant information sheet for experiment 1 at PAS lab ............................. 222

Appendix L Participant information sheet for experiment 2 ................................................ 225

Appendix M Recruitment email ........................................................................................... 228

Appendix N Education Queensland approval ...................................................................... 229

Appendix O QUT Human Research Ethics committee approval ......................................... 232

Appendix P Arrangements in Monkey Blocks given to children in Experiment 2 .............. 234

Embodied intuitive interaction in children vii

List of Figures

Figure 1 Physical-Virtual product continuum ............................................................ 17

Figure 2 Comparison of Intuitive Interaction Continua, shown through blue arrows (adapted from Blackler & Hurtienne, 2007) .................................... 45

Figure 3 Design aspects of Embodiment derived from cognitive aspects ................. 82

Figure 4 Research Design to investigate children’s Embodied intuitive interaction .................................................................................................... 88

Figure 5 XY plot of power for a range of sample size (N) value, 1 to 100 ............... 93

Figure 6 XY plot of power for a range of sample size (N) value, 1 to 100 ............... 93

Figure 7 Coding scheme showing theme groups and the corresponding sub-themes .......................................................................................................... 99

Figure 8 Physical Jenga toy (left), and virtual Jenga app (right) ............................. 110

Figure 9 Swiping and tapping to remove the block (above); placing the block on the top of the stack (below) ................................................................... 111

Figure 10 Colour-coded warnings of the danger of crashing the stack.................... 111

Figure 11 Children playing with the physical toy .................................................... 114

Figure 12 Children playing with the virtual interface .............................................. 114

Figure 13 Coding environment in Observer XT ...................................................... 115

Figure 14 Comparison of Number of Intuitive Interactions; Number of Layers Added; and Latency to decide for physical and virtual Jenga ................... 121

Figure 15 Box plot of use of aspects of Embodiment in physical and virtual toys ............................................................................................................. 124

Figure 16 Pair-wise comparisons of aspects of Embodiment for interactions with physical toy (left) and virtual toy (right) with mean ranks for each aspect ................................................................................................. 126

Figure 17 Osmo setup and Newton app ................................................................... 133

Figure 18 Children playing Osmo. The view of the tablet screen (on the left) shows Newton game in action and the view of children manipulating objects and drawing in the physical space (on the right) ........................... 134

Figure 19 Three types of blocks in Monkey Blocks: orange blocks with weights at one of the ends, green blocks with weights in the middle, blue blocks with no weights. .............................................................................. 135

Figure 20 Blocks and monkeys in arrangements ..................................................... 135

Figure 21 Example of an arrangement: black and white image given to children (left), equivalent coloured image (right) .................................................... 136

Figure 22 Number of intuitive, non-intuitive, and partially-intuitive interactions for the TEI Osmo ....................................................................................... 144

viii Embodied intuitive interaction in children

Figure 23 Intuitive, non-intuitive, and partially-intuitive interactions in the physical toy, Monkey Blocks ..................................................................... 145

Figure 24 Relative contributions of aspects of Embodiment to intuitive interaction with TEI game .......................................................................... 148

Figure 25 Relative contributions of aspects of Embodiment to intuitive interaction with Monkey Blocks ................................................................ 148

Figure 26 Artefact-Artefact Affordances: stack-ability and slide-ability of blocks in Monkey Blocks (above); pull-ability and slide-ability of blocks in Jenga (below) .............................................................................. 159

Figure 27 Research results incorporated into an Enhanced Framework of Intuitive Interaction (EFII) (adapted from Blackler et al., 2018) .............. 161

Figure 28 Embodied Cognition as distributed perceptual systems, (adapted from Gaines (1989) and Hinton (2014)) .................................................... 167

Figure 29 Perception side of the Model for Embodied intuitive interaction (MEII) in children ...................................................................................... 168

Figure 30 Actions side of the Model for Embodied Intuitive Interaction (MEII) in children. ................................................................................................. 170

Figure 31 Arrangements for Monkey Blocks game in black and white and colour ......................................................................................................... 234

Embodied intuitive interaction in children ix

List of Tables

Table 1 The type of products on the physical-virtual continuum .............................. 19

Table 2 Summary of Experiment 1 ............................................................................ 89

Table 3 Summary of Experiment 2 ............................................................................ 90

Table 4 Dependent for measures of intuitive interaction, successfulness and aspects of Embodiment .............................................................................. 120

Table 5 Descriptive statistics for Number of intuitive interactions, Number of layers added and Latency to decide corresponding to Type of Toy: Physical and Virtual Jenga ......................................................................... 121

Table 6 Mean Rank Values of Number of Intuitive Interactions, Number of Layers Added, and Latency to decide for each type of toy ....................... 122

Table 7 Mann-Whitney U Test Statistic of Number of Intuitive Interactions, Number of Layers Added, and Latency to decide for each type of toy ..... 122

Table 8 Descriptive statistics for Number of uses of aspects of Embodiment for physical and virtual Jenga .......................................................................... 124

Table 9 Mean Rank Values of Number of Intuitive Interactions, Non-Intuitive Interactions and Partially Intuitive Interactions ......................................... 146

Table 10 Regression coefficients and VIF values for the MRS system ................... 147

Table 11 Correlations between aspects of Embodiment (predictors) ...................... 149

Table 12 Strength of agreement based on ICC values, adapted from (Koo & Li, 2016) .......................................................................................................... 207

Table 13 Differences between assumptions for parametric and non-parametric data analysis methods, adapted from Field (2008) .................................... 208

Table 14 Thresholds for interpreting effect size, adapted from Cohen (1992, p.40) for correlation and Rosenthal & Rosnow (1991, p.361) for Mann Whitney U test. .......................................................................................... 209

Table 15 Coding heuristics for Types of Interaction ............................................... 210

Table 16 Coding heuristics for Aspects of Embodiment ......................................... 211

x Embodied intuitive interaction in children

Statement of Original Authorship

The work contained in this thesis has not been previously submitted to meet

requirements for an award at this or any other higher education institution. To the

best of my knowledge and belief, the thesis contains no material previously

published or written by another person except where due reference is made.

Signature: QUT Verified Signature

Date: October 2017

Embodied intuitive interaction in children xi

Acknowledgements

This thesis would not have been possible without the continued support,

mentorship and advice from my supervisors, Associate Professor Thea Blacker and

Professor Vesna Popovic. I have received immense support and advice for which I

am very grateful. Your continued enthusiasm, vision, and patience during this

process have been invaluable. I thank you very much for that. I also thank Mr. Ray

Duplock and Denise Scott. The assistance and knowledge you have both contributed

to this work has been of great importance.

Children and parents who were kind enough to help me out in the study,

without your help, it would not have been possible. I thank all my colleagues from

the People and Systems Lab (PAS lab), Creative Industries Faculty (CIF), Science

and Engineering Faculty (SEF), QUT-wide and world-wide. There are many to

thank individually, but specific mentions go out to Marianella Chamorro-Koc,

Marissa Lindquist, Deborah Fels, Mini Suresh, Shayne Beaver, Levi Swaan and

Mitch McEwan: You all have supported me at different times and in different ways.

I cannot thank you enough. Specific mention goes to Helena Papageorgiou from

Creative Industries Higher Degree Research support team, thank you for all the help

and for being patient with my constant demands and queries.

I am very thankful to the entire research community for your constructive

feedback and interesting discussions, it has greatly helped me in putting this thesis

together. I thank all reviewers and examiners for your invaluable feedback.

To my students for their invaluable and interesting discussions which have

informed my research and my knowledge and understanding of other areas of Design

Research, I thank you all.

Lastly, my family for all the love and support, I thank you. To my daughter

Vishakha, I hope this journey of mine will inspire you to reach greater heights and

goals in your own life. To my husband, Gaurang, I thank you for the strength,

patience, and encouragement all through this journey. Lastly, love and affection

shown by my furry mate, Archie, in difficult times meant a lot to me.

xii Embodied intuitive interaction in children

Embodied intuitive interaction in children 13

Chapter 1: Introduction

This chapter introduces Embodiment and intuitive interaction, and the research

problem pertaining to children’s Embodied intuitive interaction. Key concepts and

terms relevant to the study are defined in Section 1.1. The physical virtual

continuum that is used to discuss the range of products used by children is discussed

in Section 1.2. Section 1.3 then discusses interaction modalities in children as ways

in which they interact with interfaces and products. Section 1.4 discusses the

research problem, and identifies the gaps in research that lead to the research

questions. The objectives of this study are stated in Section 1.5, and the significance

of its findings is detailed in Section 1.6. Finally, Section 1.7 provides an overview of

the thesis.

1.1 Definitions of concepts and terms

Embodiment is described as an idea that an organism’s sensory-motor

capacities, body and environment not only play an important role in cognition, but

the manner in which these elements interact enables particular cognitive capacities to

develop and determines the precise nature of those capacities (Clark, 1997; Cowart,

2004; Dourish, 2001).

Cognitive processes in Embodiment rely on perception action couplings

(Gibson, 1979) which is discussed in Chapter 2 and Chapter 4 in detail. Perception

action coupling is a circular relationship where perception drives action which in turn

aids perception. The perception identifies and uses invariant information in the

environment. The action component involves movement and control features being

set and regulated to achieve an action goal.

Intuitive interaction is a subconscious application of one’s prior knowledge,

and this knowledge is derived from various sources. Familiarity with features of

products, or prior experience, is one of the sources of prior knowledge in intuitive

interaction (Blackler, Popovic, & Mahar, 2010). The definition of intuitive

interaction thus draws upon prior knowledge and its subconscious application:

Intuitive use of products involves utilising knowledge gained through other gervaisexperience(s). Therefore, products that people use intuitively are

14 Embodied intuitive interaction in children

those with features they have encountered before. Intuitive interaction is fast and generally non-conscious, so people might be unable to explain how they made decisions during intuitive interaction. (Blackler et al., 2010, p2)

Researchers at the Intuitive Use of User Interfaces (IUUI) research group in

Germany—Hurtienne & Blessing, 2007; Israel, Hurtienne, Pohlmeyer, Mohs,

Kindsmüller, et al., 2009; Mohs et al., 2006; Naumann et al. (2007)—defined

intuitive interaction as a relationship between a person and an artefact, rather than as

an attribute of an object: “A technical system is intuitively usable if the user’s

subconscious knowledge leads to effective interaction.” Embodiment and intuitive

interaction are discussed in further detail in Chapter 2, Chapter 3 and Chapter 4.

Interactions with interfaces and systems integrated and/or embedded with

technology are grouped under the common umbrella term Tangible Embodied and

Embedded Interfaces (TEIs). The broad spectrum of products under TEIs is evident

from the recent call for papers at the TEI 2017 conference (TEI 2017, 2017). The

conference asked for submissions on wearables; products that bridge physical and

digital worlds; shape-changing displays; and on the role of physicality in Embodied

sense making and tangible interactions. As advances in technology progress, and the

uses of TEIs in interaction design are explored, new configurations of TEIs will

continue to emerge. Papers presented at past TEI conferences have variously

discussed TEI systems as ‘Tangibles’ (Gervais, Frey, Gay, Lotte, & Hachet, 2016);

‘Embodied interfaces’ (Malinverni, Ackermann, & Pares, 2016); ‘Embedded

systems’ (Yoon, Huo, & Ramani, 2016); and ‘Mixed reality systems’ (Robert,

Wistorrt, Gray, & Breazeal, 2011).

‘Tangibles’ are defined as things that are “capable of being perceived

especially by the sense of touch” (Merriam-Webster, 2004). Tangible User Interfaces

(TUIs) was the term introduced by Ishii (2008) to represent objects that embed digital

information so that it can be directly manipulated and investigated. As products and

appliances became more intelligent, tangible interfaces gained popularity in

disciplines such as Industrial Design, Product Design, and Interactive Arts

(Djajadiningrat et al., 2004). Hornecker & Buur (2006) used the term ‘tangibles’ to

describe the meaning of tangibility in disciplines other than Human Computer

Interaction (HCI). They described tangibles as user interfaces and interactions that

draw on the tangibility and materiality of the interface, the physical Embodiment of


data, whole body interaction, and the Embodiment of interface and user interaction in

real spaces and contexts. ‘Tangibles’ are thus defined as

Systems that rely on Embodied interaction, tangible manipulation, physical representation of data, and embeddedness in real space. (Hornecker & Buur, 2006, p. 1)

‘Embedded interfaces’ refer to systems with computing elements with a

dedicated function, often with real-time computing constraints (Heath, 2002, p2).

The computing elements control the behaviour and operations and, in some cases,

even the look of the systems in which they are embedded. ‘Embodied interfaces’

refer to sensory interfaces that involve Embodied awareness of the human body and

the physical world (Moeller & Kerne, 2012).

Mixed reality systems are TEIs consisting of both physical elements and

virtual elements (Milgram & Colquhoun, 1999). They were developed to allow users

to take advantage of the continuous innovations in technology. Mixed reality

systems commonly referred to as Augmented systems, could either consist of real

physical world augmented with virtual objects, as in Augmented Reality (Azuma et

al., 2001), or the virtual world augmented with real physical objects, as in

Augmented Virtuality (Regenbrecht et al., 2004).

The primary differences among the TEIs discussed above are the objectives

and functionalities that they were designed for. For example, Mixed Reality systems

were developed to create immersive environments with technology. TUIs were

intended to provide a way of interacting with digital information, using various

interaction modalities. TEIs, in general, differ in their system configurations,

depending on the way their physical and virtual elements are configured and coupled.

There is ongoing research into the use of TEIs for children (Enyedy et al., 2017;

Fan, Antle, & Cramer, 2016). That research is, however, mostly focussed on the use

of various TEIs in the context of children’s learning. There is some limited research

on embedded systems in the context of children’s play (Sakai & Sugano, 2016), such

as embedding a tracking device in stuffed toys. TEIs are further discussed in the

context of Embodiment and children’s intuitive interaction in Chapter 2 and Chapter

3.

Affordances are the possibilities of actions that can be performed on an object

or environment (Gibson, 1979). Children should be able to discover these possible


actions through the interpretation of clues (Dotov, Nie, & De Wit, 2012). Inherent

properties of objects and environments such as shape, material, size, and colour

could offer clues for possible actions (Harrison & Hudson, 2009; Kim, 2015;

Norman, 1999). For example, children could interpret a soft and squishy object as

something to be pressed and squeezed; or a spherical object could be interpreted as

something to be rolled or bounced. Since these clues are natural to the objects, they

are referred to as ‘natural clues’. However, clues could also be deliberately

incorporated into the design of an object to facilitate the discoverability of its

possible operations; these clues are referred to as ‘deliberate clues’ (Norman, 1999).

The absence of such clues could result either in incorrect actions (actions that are

unintended or unexpected), or no action with the objects at all. Gaver (1991) referred

to such affordances as ‘false affordances’ and ‘hidden affordances’, respectively.

Thus, the term ‘clues’ in this research means

a set of features that either exist naturally in objects and environments or incorporated deliberately in design to facilitate discoverability of possible actions and operations on the objects and environment.

The most common forms of deliberate clues are symbolic in nature, and have a

definite meaning. The understanding of such deliberate clues is developed through

experience and/or learning of the meaning of the symbols in the same context of use,

or from another context of use. These deliberate clues are referred to as ‘symbolic

clues’. Another form of deliberate clues, suggested by Pezzulo (2011), is called

‘Embodied representations’. These are re-enactments of motor processes in the mind

that are generated by watching something in action, rather than by performing

actions oneself. In other words, Embodied representations are action-based clues that

trigger the same neural processes of action and perception as does the performance of

those actions. Animated icons and simulations of objects depicting their functionality

are examples of Embodied representations. Natural and deliberate clues are further

discussed in relation to affordances in Section 5.4.1.

When interpreted and acted upon, clues, both natural and deliberate, could

reveal another clue (or clues) for further possible action. Gaver (1991) refers to such

affordances as ‘sequential in time affordances’. Similarly, affordances could be

nested in space; that is, more than one clue is available at different places to facilitate

the perception of possible action (or actions) with an element.


1.2 PHYSICAL-VIRTUAL CONTINUUM

Children have access to products that range from those with physical elements

only, to those with virtual elements only. These products can be placed on a

physical–virtual continuum (as shown in Figure 1). The physical environments, on

the left, define products and interfaces consisting solely of real physical objects;

these include objects such as blocks, sticks, pens, and paper. These objects lack

computing elements, and people mostly rely on their sensorimotor knowledge to

interact with them. The virtual environments, on the right, define products and

interfaces consisting solely of virtual elements; these include conventional computer

graphic simulations, and apps on a screen (such as a monitor, tablet, or phone).

Figure 1 Physical-Virtual product continuum

Anything between the two extremes of the continuum are TEIs, that is, systems

or interfaces with both physical and virtual environments, where both real world and

virtual world objects co-exist. TEIs towards the extreme left of the continuum would

allow more interactions with the physical elements, and those towards the extreme

right would allow more interactions with the virtual elements.

1.3 INTERACTION MODALITIES

Children’s actions with artefacts are the result of the simultaneous deployment

of various semiotic resources, such as speech and body gestures, to convey

information (Goodwin, 2000). Streeck (2013) and Streeck, Goodwin, & LeBaron

(2011) further added that children’s interactions with the physical, material, and

social world also contribute to Embodiment. Thus, interaction modalities—that is,

the ways in which children carry out actions with products, interfaces, and the social

world—have been explored for Embodied interaction. Traditional interaction

modalities have limited children’s interactions to tactile and visual. However, with

the integration of technology in TEIs, new interfaces with various interaction

modalities, that accept physical interactions (gestures, touch, and body movements)

as inputs to digital environments, have further strengthened the possibility of


incorporating Embodiment with technology in children’s learning and playing (van

den Hoven & Mazalek, 2011).

Franinović & Serafin (2013) discussed the use of sound as an interaction

modality. Fink et al. (2014) studied the use of robots in encouraging children to tidy

their rooms. They developed robotic toy boxes that initiated an interaction by

encouraging children to pick up toys after play and put them in the box. The toy box

then provides auditory and visual feedback to the children. For example, Bubu

Monstry makes chewing and eating sounds when a toy is placed in the box; when all

toys are back in the box, Bubu vibrates and burps. Bubu‘s moving eyebrows and

eyes, provides positive visual feedback. Other interaction modalities with TEIs are

full body interactions, gestural interactions, and direct manipulations. Full body

interaction is the use of full-body movement as an interaction modality for physical

and digital artefacts. Gestural interactions allow individuals to foster bodily and

physical engagement with an artefact (Aslan, Primessnig, Murer, Moser, &

Tscheligi, 2013).

Natural user interfaces (NUIs) allow interactions in ways that are more like

those used to interact with people and objects in the real world. NUIs include natural

forms of interaction, such as bodily movements (Hoffmann, Schuster, Schilberg, &

Jeschke, 2016); spatial gestures (Hay, Newman, & Harle, 2008); facial expressions

(Veeriah, Pilarski, & Sutton, 2016); speech (Dahl, 2017); and touch (Anthony,

Stofer, Luc, & Wobbrock, 2016). Because they are natural, NUIs are thought to be

natural to use, and require very little training and learning. Fitts’s law has played an

important role in Human Computer Interaction (HCI), and is often used to design and

evaluate NUIs for ease of use. The law states that the time of completion of an

interaction is a function of distance, and the size of the target which is acted upon

(Fitts, 1954). Time of completion is very important in interaction design, as the

longer it takes a user to navigate an interface, the higher the possibility of their

becoming frustrated with, or even giving up on the interface. Thus, NUIs that are

designed for faster time of completion are interfaces that excel in terms of

learnability. Interactions with NUIs require knowledge that users derive from their

everyday lives. Thus, NUIs exploit metaphors with real-world objects to facilitate

interaction with digital objects in the digital world.


Gestural interactions and NUIs were popularised in futuristic movies such as

Minority Report (Arthur, 2010; Dick, 2002) and Mission Impossible (Cruise,

Wagner, & Brian, 1996) and, in the children’s context, have been popular in gaming

environments such as XBOX Kinect (Microsoft, 2010), and interactive tables and

walls (Anthony et al., 2016). Recent research efforts have focussed on the role of

gestural interactions and NUIs in Mixed reality systems (Santos, Cardoso, Ferreira,

Ferreira, & Dias, 2016). There could be overlaps between full body interactions and

gestural interactions, as a combination of both is often used in NUIs (Macaranas,

2013).

However, Mistry & Maes (2009), Norman (2010) and Shaer & Hornecker

(2010) suggested that gestural interactions are neither easy to learn nor remember.

Some gestures can be confusing as they differ from one culture to another, and the

derivation of meaning from gestures and body movements relies heavily on an

individual’s experience and knowledge. Norman (2010) cites Indian head shake and

hand waving gestures as some of these confusing gestures. Furthermore, gestural

interactions do not leave behind any traces of interaction. Thus, if an interaction does

not result in an appropriate response, it is impossible for an individual to determine

the cause of the failure.

Direct manipulations with tactile interactions are an alternative to body and

gestural interactions. They eliminate the coupling issues between the user’s

interaction and the artefact. Children are familiar with tactile interactions from birth,

and through their developmental stages as they get older. Thus, tactile interactions

are the most common modes of incorporating Embodiment in children’s learning

activities, especially in the early years (Robins & Dautenhahn, 2014). Direct access

to the spatial and material properties of the elements and features that are touched,

allows children to provide meaning to their interactions.

This research studied children’s Embodied intuitive interaction, using tactile

interactions as an interaction modality. To this end, it explored three types of

products, as identified on the physical-virtual continuum: physical, virtual, and TEIs

(Table 1).

Table 1 The type of products on the physical-virtual continuum


1.4 RESEARCH PROBLEM

In their everyday lives, children are increasingly expected to learn to use

complex concepts, devices, and products. As these concepts and products become

increasingly complex, they become more difficult to understand and use. The

essential criteria for usability of a device is that its operation should be easy to learn,

and that children can, either consciously or subconsciously, anticipate its behaviour

(Hartson, Andre, & Williges, 2003).

Embodied interactions result in experiences that are engaging, magical, fun,

emotional, and affective (Fincham, 2016; King & Chang, 2016). Wensveen et al.

(2004) suggested that children’s intuitive interactions with products and interfaces

result in positive experiences. Israel, Hurtienne, Pohlmeyer, Mohs, Kindsmuller, et

al. (2009) suggested that knowledge from Embodied interactions with the physical

world contributes to intuitive interactions. Blackler & Hurtienne (2007) suggested

that the simplest form of intuitive interaction is based on Embodied knowledge

learned early in life. There is an increasing push to incorporate Embodiment into

children’s learning (Björk-Willén & Cromdal, 2009; Breathnach, O’Gorman, &

Danby, 2016; Whitebread, Basilio, Kuvalja, & Verma, 2012). Embodied play and

Embodied learning are powerful forces in children’s interactions with the world

(Whitebread et al., 2012).

An Embodied view of cognition could contribute immensely to an

understanding of how Embodiment impacts and influences intuitive interaction.

Nevertheless, the bodies of literature on intuitive interaction and Embodied

interaction are largely independent of each other. For example, Hummels, Smets, &

Overbeeke (1998) discussed the possibilities that gestures could offer to support

intuitive human-computer interfaces for product design; however, their study was

Type of Product

Interaction modality Physical virtual continuum

Product/Interface used in this study

Physical Tactile interaction with the physical elements

Extreme left of the continuum

Jenga blocks Monkey Blocks

Virtual Tactile interaction with the virtual elements

Extreme right of the continuum

Jenga app

TEIs Tactile interaction with the physical elements and visual interaction with the virtual elements

Middle of the continuum, towards the Physical, left end of the continuum

Osmo with Newton app


focussed on gesture, while the meaning of Embodiment was limited to the hand

movements used in gestures.

Embodiment and Embodied interactions are considered to offer natural and

intuitive form of interactions. This claim has been made in literature, few studies

such as Antle, Corness, & Droumeva (2009a), discuss the cognitive structures of

intuition underlying Embodiment. Research in Embodied interaction design has

emphasised the affordances offered by body movements in interacting with a

product, and has described Embodiment in relation to the role of body and space in

interaction design. This has led to the use of the term ‘Embodiment’ to mean bodily

action or physicality (Ahmet, Jonsson, Sumon, & Holmquist, 2011; Grufberg &

Jonsson, 2012; Holmquist et al., 2010; Johansson et al., 2011; Schafer et al., 2013;

Søndergaard, 2013).

There is more to Embodiment than simply physical body movements.

Hurtienne, Löffler, Gadegast, & Hußlein (2015), and Macaranas, Antle, & Riecke

(2015) suggested the use of metaphors in interaction. The meanings of these

metaphors are derived from people’s Embodied knowledge of the world, or their

sensorimotor knowledge. To support intuitive interaction, interfaces can be designed

with interactional metaphors based on abstract representations with physical body

movements as means to incorporate those metaphors. Antle & Wise (2013) studied

an interactive musical sound-making environment that uses an Embodied metaphor

to map body-based input with audio output. In their empirical experiments, some

children (aged 7 to 10 years) were asked to perform specific sound sequences by

varying a single sound parameter (volume, tempo, pitch, and rhythm) associated with

specific physical movements, while others used a system without Embodied

metaphors. They found that the Embodied metaphor-based system was more intuitive

than the system without the Embodied metaphors. The experiments have proved the

effectiveness of Embodied metaphors in the intuitive use of interfaces. However,

further and ongoing research needs to focus on the appropriateness of Embodied

metaphors for children, as the discoverability of metaphors depends on children’s

past experience and knowledge which, when compared to an adult’s, is quite varied

(Bakker, Van Den Hoven, & Antle, 2011).

Direct manipulation and interaction with objects through tactile or haptic

interactions rely on Embodied interactions (Hornecker & Buur, 2006). Haans &


IJsselsteijn (2006) suggested that interacting with the environment around us through

touch offers a phenomenal extension of the self, enabling genuine Embodied

interaction. They further added that tactile interactions could offer Embodied

interactions with technology, eventually blurring the boundary between

“…unmediated self and the mediating technology...” (p156). Thus, directly

manipulated tactile interactions are one of the modalities used in Embodied

interaction. Being intuitive is attributed to touch-based interactions—tactile or haptic

interactions with static system properties such as being direct (Dix, 2011), easy to

learn, natural (Muller-Tomfelde & Fjeld, 2012), fast, simple, and effective (Jacoby et

al., 2009). However, few studies have attempted to empirically prove the validity of

the intuitive claims for tactile interactions with physical products. Tactile interactions

with TEIs can become complex as they often involve interactions at different places

or nodes on the product, and the corresponding feedback from the product is also at

different places/nodes (Wensveen, Overbeeke, Djajadiningrat, & Kyffin, 2004).

This research investigated Embodied intuitive interaction in three types of

interfaces and systems, as highlighted on the physical virtual continuum (Figure 1) -

physical products, TEIs and virtual interfaces—with tactile interaction as an

interaction modality.

Research in intuitive interaction has mostly focussed on adults (Blackler et al.,

2010; Lawry, Popovic, & Blackler, 2011) and adults in contexts such as airport

navigation (Cave, Blackler, Popovic, & Kraal, 2014) and web interaction (Mohan,

Blackler, & Popovic, 2015).There has been limited research into children’s intuitive

interaction (Desai, Blackler, & Popovic, 2015, 2016). Although there is ample

research into Embodied interaction in children (Marshall et al., 2009; Montemayor et

al., 2002; Sherman, Druin, & Montemayor, 2001; Zaman, Abeele, Markopoulos, &

Marshall, 2009), there is limited research that investigates Embodiment and intuitive

interaction in children (Antle, Droumeva, & Corness, 2008). To address this lack,

this study investigated the role of Embodiment in intuitive interaction through

children’s tactile interaction with physical products, virtual interfaces and TEIs It

investigated how children use their environment and social interactions to inform

their mind and body when interacting with an interface or a product. The study was

supported by two empirical experiments, as detailed in Chapter 6 and Chapter 7.


Specifically, the study focussed on the following research question and three sub-

questions:

What is the role of Embodiment in children’s intuitive interaction?

i. Can Embodiment facilitate intuitive interaction in children?

ii. Which design aspects of Embodiment facilitate children’s intuitive

interaction

iii. To what extent do the design aspects of Embodiment facilitate

intuitive interaction in children?

Design aspects of Embodiment were derived through the literature review of

Embodied cognition (Chapter 4) and were then used in Experiment 1 and Experiment

2 to address the three sub-questions.

1.5 AIMS AND OBJECTIVES

The aim of this research was to gain an understanding of aspects of

Embodiment and the role that they play in children’s intuitive interaction. Using

children’s play with toys as the context, the research objectives were to:

Understand the cognitive aspects of Embodiment in the literature, and

study its design aspects to facilitate children’s Embodied intuitive

interaction

Investigate children’s Embodied intuitive interaction with physical

products, a virtual interface (app), and TEIs

Investigate the influence of the design aspects of Embodiment on

children’s intuitive interaction when playing with physical toys, virtual

interface, and TEIs

Develop an interaction model that represents the use of Embodiment in

children’s intuitive interactions with physical and virtual products and

TEIs.

1.6 RESEARCH SIGNIFICANCE

This research makes a number of significant contributions through its

generation of new knowledge and its outcomes. New insights gained contribute to


advancing knowledge of children’s interactions with physical and virtual products

and TEIs, and in turn to advance the theory of children’s Embodied intuitive

interaction. A significant contribution is the development of an interaction model –

Model for Embodied Intuitive Interactions (MEII) that illustrates children’s

Embodied intuitive interactions with physical and virtual products, and TEIs. Further

significance of this research study is demonstrated by the transferability of the

study’s outcomes to the continua of intuitive interaction, resulting in an Enhanced

Framework for Intuitive Interaction (EFII). The following sub-sections provide an

overview of these key contributions.

1.6.1 Contribution to knowledge

There are three significant contributions of this research study: (i) It contributes

to knowledge on Embodied intuitive interactions in children with tactile interactions

as an interaction modality; (ii) It provides an understanding of the role of

Embodiment in facilitating children’s intuitive interaction; and (iii) It provides a

methodology for investigating children’s Embodied intuitive interaction.

(i) Knowledge on Embodied intuitive interactions in children with tactile

interactions as an interaction modality.

This research strengthens the understanding of aspects of Embodiment in

children’s intuitive interaction. In particular, it investigates Embodiment in

relation to tactile interactions as an interaction modality. The research study

contributes new insight into Embodied intuitive interaction, using tactile

interactions with physical products, virtual interfaces, and TEIs. This is a

significant step towards incorporating Embodiment in the design of

products on the physical-virtual continuum, as it is the nature of the tactile

interactions with products and interfaces that contributes to intuitive

interaction. Previous research mostly focussed on full body interactions and

Embodied metaphors as facilitators of children’s intuitive interaction.

(ii) Understanding of the role of Embodiment in facilitating children’s

intuitive interaction

The results of this study contribute to an Enhanced Framework for

Intuitive Interaction (EFII) (Blackler, Desai, McEwan, Dieffenbach, &

Popovic, 2018). This framework could lead to new directions in research


into intuitive interaction, such as intuitive experiences, and the factors

responsible for these experiences (e.g. transfer distance, indirection, and

ubiquity).

The transferability of the results to Blackler & Hurtienne’s (2007) continua

of intuitive interaction suggests that the findings could also be applicable

to adults. This is an important contribution because Embodied intuitive

interaction is conventionally associated with children as sensorimotor

knowledge is considered to be more accessible to children than to adults

(Brandenburg & Sachse, 2012). The applicability of the EFII to adults

could lead to future research that investigates Embodied intuitive

interaction in adults.

The EFII allows discussion of intuitive interaction with products on the

physical-virtual continuum: physical products, virtual interfaces, and TEIs.

The findings from the research study concur with Blackler's (2008)

findings that physical affordances should be incorporated in design

whenever possible. Where physical affordances are not possible, perceived

affordances should be used. The study has further suggested in form of

recommendations that physical and perceived affordances could be used

together, sequential in time, or/and nested in space. This study thus has

implications for design, and provides design recommendations for

Embodied intuitive interactions with physical products, virtual interfaces,

and TEIs for children.

This study further highlights the role of cooperative activity and

scaffolding as Embodied aspects in the design of children’s products.

Appropriate scaffolds can assist children in offloading cognitive activity

onto the environment around them while carrying out perceptual motor

activities. Emergence is an aspect of design that is representative of

dynamic processes in systems where interactions, behaviours, and

environments evolve over time. This study has suggested that dynamic

processes are conducive to Embodied intuitive interactions as they

facilitate the updating of existing knowledge and the generation of new

knowledge; in other words, learning of new concepts. This contribution is


significant because it provides guidelines for the design of Embodied

intuitive products (physical, virtual, and TEIs) for children.

(iii) Methodology to investigate children’s Embodied intuitive interaction

This research contributes a novel methodology for the study of aspects of

Embodiment and children’s intuitive interaction. The context of play and

co-discovery between two children in an observational setting allowed data

to be collected, with little disruption to the children’s cognitive processes.

Retrospective interviews (performed after observations) provided

clarification and a better understanding of the use of aspects of

Embodiment in intuitive interaction. This methodological contribution is

significant as it can be transferred to other application domains involving

children, such as spatial interactions.

1.6.2 Research Outcome

The findings from this research study have contributed to the model for

Embodied Intuitive Interaction (MEII)—an interaction model that represents

children’s behaviour with artefacts and the social world. It illustrates that children are

distributed anticipatory systems; that is, they use their sensory perceptual systems,

sensorimotor knowledge, and Embodied experiences to interact with artefacts and the

social world. They also use physical affordances, perceived affordances, and

scaffolding in cooperation with other children to perceive the clues in the social

world and the artefacts, to decide on the actions to be performed.

Having decided on these actions, they use scaffolding in cooperation with other

children to perform the actions on artefacts and the social world. While these actions

are being performed, however, the properties of the artefacts and the social world

change. Thus, over multiple perception and action cycles over time, the artefacts,

social world, and children’s interactions evolve, resulting in ‘emergence’. This

dynamic nature of Embodied intuitive interaction is important for children’s

development of new knowledge, and for the updating of their existing knowledge

and learning. This is a significant contribution of the study as it offers insights into

how children interact with their physical and social world. MEII (Figure 29 and

Figure 30) could be applied in various contexts relating to children, such as

designing interactions for children. This model will allow designers to evaluate and


design interactions for Embodied intuitiveness, and will inform the further design

and development of tools to design Embodied intuitive products for children.

1.7 THESIS OVERVIEW

Chapter 2 reviews the literature on intuitive interaction, and on the nature and

processes of Embodiment in the context of cognition and interaction design. Chapter

3 discusses Embodied intuitive interaction in the context of children as user groups.

Chapter 4 addresses the first part of the research question, ‘What are the

aspects of Embodiment?’ It discusses the aspects of Embodiment in cognitive

science and derives design aspects of Embodiment which were then used in

Experiment 1 and Experiment 2 to investigate Embodied intuitive interaction in

children.

Chapter 5 describes the research design and methodology, and justifies the

methodology according to the needs of the research question and the available

methods. The two experiments are briefly explained. Heuristics for intuitive

interaction, and design aspects of Embodiment derived from the respective literature,

are explained.

Chapter 6 and Chapter 7 describe the two empirical experiments, Experiment 1

and Experiment 2, respectively. The results are described and discussed from the

perspective of implications to design. Chapter 8 discusses the results of Experiment 1

and Experiment 2 in reference to the continuum of intuitive interaction. The

contribution of this study to the Enhanced Framework of Intuitive Interaction (EFII)

(Blackler et al., 2018) (Figure 27) is discussed. The outcome of this study is The

Model of Embodied Intuitive Interaction (MEII) which is discussed in Chapter 8.

Chapter 9 highlights the contributions of the study, and ends with a statement of the

future scope of the research.

1.8 SUMMARY

Chapter 1 has introduced the research study with a brief background on the role

that Embodiment and intuitive interaction could play in children’s interaction with

complex products. Key concepts and terms relevant to this study were defined and

explained. A physical virtual continuum was used to discuss the range of products

that children use, and how they vary in their physicality and virtuality. Interaction


modalities that form the basis of children’s interaction with products were discussed.

A discussion on the research problem identified research gaps leading to the research

questions. Aims and objectives of the study were defined. Finally, the chapter

outlined the contributions made by this research study to new knowledge, and the

outcomes of the research were discussed.

Chapter 2 now reviews the literature on intuitive interaction and the nature and

processes of Embodiment, in the context of cognition and interaction design.

Literature on intuition and intuitive interaction is discussed. Different perspectives of

Embodiment based on the role of brain, body and environment in the cognitive

processes of Embodiment are discussed.

Chapter 2: Embodiment and Intuitive Interaction

This thesis brings together two important frameworks in interaction design,

Embodiment and Intuitive Interaction. To this end, this chapter reviews the literature

on: i) Embodiment (Section 2.1)—the alternative to the traditional theory of

cognition, and considers that the brain, body, and the environment work together to

process any stimuli; ii) Embodiment in interaction design (Section 2.2)—interaction

design that is based on the concept of Embodiment; iii) Intuition and intuitive

interaction (Section 2.3)—intuitive interaction in design from the perspective of

cognitive psychology; and, finally, the research on iv) Embodied intuitive

interaction—that discusses the relationship between Embodiment and intuitive

interaction, and the limited research carried out in the field of interaction design.

2.1 EMBODIMENT

The basic idea of Embodiment is that the brain alone should not be treated as

the only place where mental processes occur. The body and its movement, guided by


perceptual coupling with the environment, facilitate the goal achievement, thus could

eliminate the need for complex internal mental representations. Embodied theories of

cognition present different principles by which aspects of perceptual and motor

processes are tightly coupled, not only to each other, but also to higher order

cognitive processes, including language (Clark, 2008) and mathematics (Barsalou,

Santos, Simmons, & Wilson, 2008).

The theory of Embodiment contradicts the traditional view on cognition that

suggests the separation of mind, body, and environment. The traditional view of

cognition has been discussed in various fields and has been referred to by different

names: Cartesianism or Dualism in cognitive theory (Husserl, 1960/2013); and

Brainbound (Haugeland, 1989) in Artificial Intelligence. Descartes (1911, 1985)

suggested that the mind is the locus of intelligence, and the body is simply a sensing

tool. The concept of Brainbound suggests that the body is just the sensor and effector

system of the brain, and that the rest of the world is the arena in which adaptive

problems are posed, and the brain-body system must sense and act (Haugeland,

1998). Traditional theories of motor development are based on the belief that

movements are the result of commands from the brain, and that all movements result

in actions, and that repetition of movements leads to action learning. Embodiment,

on the other hand, rejects the understanding that intelligence is the most important

feature of the notions of thought and reason.

Researchers in Artificial Intelligence have been investigating whether

machines can be designed to think like human beings. Turing (1950) formulated a

test, popularly known as the ‘Turing test’, to determine whether a machine can do

what humans, as thinking entities, can do. In an attempt to develop machines that

think and act like humans, machines were formulated as entities that manipulate

abstract representations by explicit formal rules. These rules came to be known as

GOFAI (Good Old Fashioned Artificial Intelligence). Searle (1980) was critical of

GOFAI and argued that machines could pass the Turing test simply by manipulating

symbols of which they had no understanding. Machines without appropriate

understanding of those symbols could not be considered to be thinking like humans.

Cartesianism explains functions of mind such as thinking in terms of symbolic

manipulations according to explicit rules (Vogt, 2002). Internal representations,

formal abstractions and rule based transformations are used to manipulate symbols. It


is the form of symbol and not its meaning that forms the basis of the rule based

transformation (Anderson, 2003) in Cartesian cognition. The relation between sign

and signifier seems arbitrary and this creates a distance between the inner symbol

processing and the external world of meaning and action (Harnad, 2003). Rule based

transformation outlines specific rules for thinking. Having disconnected the form of

symbol from its meaning, Cartesianism rules out the possibility of content sensitive

processing, and so requires formal rules to govern the transformation from one

cognitive state to another (Clark, 2012). Physical Grounding is about how abstract

representations can acquire real-world meaning and centrally involves understanding

of how cognitive contents (however these are ultimately characterised, symbolically

or otherwise) must ultimately ground out in terms of the agent’s embodied

experience and physical characteristics (Gallese & Lakoff, 2005; Harnad, 2003;

Vogt, 2002).

Piaget (1955/2013) investigated how, and when, children develop an

understanding and knowledge of objects and properties. He asked children of various

ages to look for a toy hidden behind an obstacle. The toy was hidden in view of the

children, so they knew where it was hidden. Piaget found that children 7 months old

and under, however, did not look for the toy, as if it did not exist. Children 12

months and over retrieved the toy, having developed an understanding that although

they could not see it, it did exist. When Piaget changed the location of the toy after

the children had successfully retrieved it several times from the first location, he

found that children reached out for the toy in the first location rather than the second.

Piaget referred to this as A-not-B error, A and B referring to the first and the second

locations.

The older children had developed an understanding that objects persist even

when they move out of view and eyesight. However, the action of reaching for the

toy was unable to use that knowledge (Piaget, 1955/2013). Baillargeon & Graber

(1988) related the error to children’s competence and performance, suggesting that

they had not developed the competence necessary to access the acquired knowledge

to reach for the toy. Thelen (2008) rejected the performance and competence link,

and explained the A-not-B error using an Embodied dynamic systems model that

they had developed through a series of experiments with children. It is the dynamics

of perception and action over time that allows children to develop an understanding


of the reaching task. The A-not-B error is due to the fact that children are expected to

remember the hidden toy every time the location is changed. However, they tend to

remember the last location where it was hidden, even though they see the toy being

hidden in a different location. Thelen (2008, p. 4) explains: “The A-not-B error is not

about what infants have and don’t have as enduring concepts, traits, or deficits, but

what they are doing and have done”.

Another popular study in the field of Embodied cognition has been to

investigate how a baseball outfielder catches a fly ball. Saxberg (1987) suggested

that outfielders perceive the parameters that determine the flight of the fly ball, such

as initial direction, velocity and angle, along with other constants such as drag, air

density, and gravity. They then use these parameters as inputs to the internal

representations of inflight motion. Thus, the outfielders are then able to estimate the

trajectory of the fly ball. Cutting & Vishton (1995) challenged this point of view,

indicating that the parameters determined by the outfielder would be erroneous, as

the changes in optical projection size of the ball is not an appropriate resource to be

used to determine where it is going and how far away it is. Bingham (1995) divided

the whole process—from the point when the ball is hit to the point when it is caught

or dropped by the outfielder—into events that unfold over time. Underlying

dynamics then provide a better understanding of the changes in the events, such as

how the changes have occurred, and what forces have caused those changes.

Dynamics of inflight motion generate kinematic information that the outfielder uses

to reach a certain location at a certain speed.

Researchers have seen the advantages of studying Embodiment in animals

(Clark, 2005; Dautenhahn, 1996). The idea is to establish that perception-action

couplings (Section 1.1) produce complex adaptive behaviour in animals, and to then

develop explanations for similar solutions and behaviours in humankind.

Researchers, for example, have studied coordinated activities in groups of animals

(Barsalou, Niedenthal, Barbey, & Ruppert, 2003). Some animals move in groups to

defend against predators. The sustainment of these groups requires coordination and

management, and this is usually not centrally controlled. There is no obvious

intention to control or coordinate the group. However, the coordination emerges from

perception-action coupling rules specific to a context. Muro, Escobedo, Spector, &

Coppinger (2011) studied coordination in wolves hunting in packs. They found that


the wolves moved towards the prey until a minimum safe distance was reached, and

then moved away from any other wolves that were also close to the prey. The entire

coordination was neither planned nor instructed by anyone, but emerged from a

perception-action coupling strategy implemented by each of the wolves in a specific

context.

The traditional approach in cognition and Artificial Intelligence has been to

rely on representations in the mind and machines to process information captured

from the environment to make subsequent decisions. However, immediate actions in

the world without internal representations are evident in many real world complex

scenarios. For example, a typical mother shuttles her children to their various

activities at the right time, in the right order, meanwhile figuring out ways to do the

laundry and grocery shopping, and pick up the dry cleaning. Similar complexity is

evident in daily conversations with others, where we need to remember what has

been said, decide what to say next in light of the overall conversational goals, correct

misapprehensions, and reinterpret past interactions in light of the corrections and,

thereby, sometimes significantly alter the current state of the dialogue (Norman,

2013; Vera & Simon, 1993).

Internal representations, however, need not be completely rejected. While the

juggling mother needs representations and symbols, she cannot rely on symbols

alone. Her representations must be highly selective, related to the context, and

physically grounded. This strongly suggests that her inherent mental and physical

powers of representation should be linked to, and constrained by those which govern

her moving and acting in a dynamic environment. Thus, the execution of complex

tasks requires both reactive and representational powers.

2.1.1 Perspectives of Embodiment

The meaning of Embodiment is not as straightforward as suggested in some

literature, “states of the body modify the states of the mind” (Wilson & Golonka,

2013, p. 1). There is lack of clarity in the claims that have been made in literature on

Embodiment. For example, Dennett (1991) argued that literature on Embodied

cognition does not discuss the influence of narratives of the past which could

influence the intentions of the future. But from Strawson's (1999) view, memories

and experiences of an individual contribute to internal representations which is

rejected by theory of Embodied cognition. Then, there are researchers who believe


that it is the intentions to meet the future goals that drive individuals to use their

memories and narratives in their present to derive information from the environment

(Giummarra, Gibson, Georgiou-Karistianis, & Bradshaw, 2008; Rapp & Kurby,

2008). Moreover, there are multiple perspectives of Embodiment based on where

cognition originates from and how are cognitive processes distributed between brain,

body and environment.

Goldman & de Vignemont (2009) suggested a minimal interpretation of

Embodiment, referred as ‘Minimalist Embodiment’, where brain, body and the

environment participate in the cognitive processes, but brain is still considered to

control all cognitive processes. They suggested that all cognitive processes originate

in the brain and the body (without the brain) just supports and augments these

processes. Chiel & Beer (1997), Shapiro (2004) on the other hand suggested that

anatomy and movement of body structures shapes cognition before and after the

brain processes the information. Anatomy and movement of body structures result in

perceptual processes which could in turn result in body movements. It could thus be

appropriate to say that cognition originates in the body, the structure, composition,

and motor abilities of which determine what humans experience in the environment.

Johnson (2010) further added to the theory of cognition originating from the

body structures, suggesting that these body structures determine what people

experience, and how they understand the world. Reasoning with abstract concepts

requires mental simulations based on concrete motor-perceptual experiences

(Barsalou, 2010). Humans think, generate ideas, and act through these experiences.

The experiences offer structure to what people perceive, how they get around in the

world, how they relate to other people, and eventually result in conceptual systems

that define their everyday activities. Lakoff & Johnson (1980/2003) suggested that

the motor-perceptual experience makes the conceptual systems metaphorical in

nature. The metaphorical nature of the conceptual systems, however, is Embodied in

nature.

Language or linguistic systems are one such conceptual system that Lakoff &

Johnson (1980/2003) used to explain the Embodied metaphorical nature of

conceptual systems. According to them, semantics arose from the nature of the body,

in the form of Embodied metaphors. The most common types of metaphors are

structural metaphors, which represent abstract complex experiences with simple,


familiar, and known experiences, as seen in the phrase ‘defend an argument’.

Orientational metaphors are mostly spatial, as they arise from the orientations of the

body and its function in the physical environment. An example of such metaphors is

the concept of happiness and sadness, and the conceptual metaphor of ‘Feeling UP or

DOWN’. Happiness is associated with being physically at a high point, while

sadness is associated with being low down, at the bottom. An ontological metaphor

represents an abstract concept as a concrete physical element such as an object,

person, space or substance (Lakoff & Johnson, 1980). For example, in the metaphor

‘Joy filled the room’, ‘joy’ is an abstract emotion that is represented as a physical

object or substance that can fill a container. This allows people to link their abstract

experiences to physical entities that can then be categorised qualitatively or

quantitatively. The essence of metaphor is experiencing one kind of concept in terms

of another, and both can be completely unrelated.

Wundt (1922/1973) referred to gestures that transfer concepts from one domain

to another as ‘symbolic gestures’, and offered the use of spatial gestures to represent

temporal concepts as an example. Cienki & Müller (2008) described the

metaphorical nature of symbolic gestures through an example of the statue of Lenin

pointing toward the ‘bright future’ of communism. Pointing ahead in space is used to

indicate that an abstract object, ‘the future’, is situated ahead in time. Space is used

to represent time, and the pointing gesture that uses space to refer to time is

considered to be a metaphoric gesture.

Another perspective of Embodiment is that of functionalism, referred as

‘Embodied functionalism’, where cognition is considered to be originating in the

brain but then extending to the body and then to the environment (Clark, 2008, p.38).

The body acts as a vehicle for cognitive processes, suggesting that the body plays

only a functional role in Embodiment. Menary (2010) sees an overlap between

‘Embodied functionalism’ and ‘Minimalist Embodiment’ and suggests that cognitive

processes and brain extend beyond the body to include elements of environment in

which an individual is embedded and the interactions of an individual with the

environment. Menary (2010) refers to this interpretation of Embodied cognition as

‘Extended Cognition’.

The enactive perspective of Embodiment, similar to Embodied functionalism,

suggests that cognitive processes are distributed across brain, body and the


environment. However, cognitive processes originate in the bodily structures through

perceptual sensory systems in the body and are distributed to the brain and the

environment. The body structures include movements and actions and anatomy, but

also complex behavioural and emotive processes such as facial expressions, postures,

gestures, and so on (Noë, 2004). The enactive perspective has overlaps with the

perspective of Chiel & Beer (1997) and Shapiro (2004) which also suggests that

Embodiment originates in the bodily structures but the way the body structures are

defined in the Enactive perspective is the topic of ongoing research.

The perspective of Embodiment used in this study is based on Gibson's (1979)

theory that organisms are able to directly perceive things around them; thus,

perception, and by extension the environment, is a useful resource in cognition.

Thus, the perspective used in this study is that cognition originates from the

environment and through a continuous cycle of perception action loops (Section 1.1)

is distributed through the body and brain (Anderson, 2003; Kirsh, 2013a, 2013b;

Wilson & Golonka, 2013; Wilson, 2002). The brain is no longer considered the

centre of all behavioural and cognitive processes. It enhances and augments the

actions and activities of the body. The individual deciphers the environment,

interprets the actions to be performed on the body and the environment, which is

again followed by perception of the environment. This continuous loop of perception

action is characteristic of a dynamic anticipatory system. Cognition involves

processes where perception contributes to most part of these processes (Barrett,

2011). Once the perception action loop is initiated, it is difficult to distinguish when

perception stops and cognition starts and vice versa (Barrett, 2011, p. 55). Shapiro

(2011) suggested that the need for the objects and processes of traditional cognition

(concepts, internally represented competence, and knowledge) is replaced by

processes that rely on dynamic systems of these perception action couplings. Shapiro

(2011) referred to this as the ‘replacement hypothesis’.

Knowledge has been typically considered to be some form of internal

representational state (Rasmussen, 1985). However, Embodied Cognition theory

describes knowledge as an entity derived from an individual’s perceptual coupling

with the environment and the body. This perceptual coupling is achieved through

extension of mental processes beyond neurological processes into the parts of the

world that are of everyday use (Clark, 2001). This extension of mental processes is


seen in everyday activities such as navigating a city using a map. Although the brain

might have information about the city, people use tools such as maps (that are parts

of the world) to find their way, or at the least to check if they are on the right path.

These activities usually result in a decision such as the direction or route to follow.

The transitions between the mind and the tools, and ultimately to the decisions taken,

are seamless.

Payne (1991) examined how information flows from device to users during

their interaction with keyboard based displays and word processors. He found that

users often did not know the effects of the frequently used actions such as where will

the position of the cursor be. Thus, users need to acquire information from the

device’s display (which is the environment in this context) using their perceptual

sensory mechanisms.

Embodiment and Embodied cognition theory laid the foundation of this study

on Embodied intuitive interaction, literature on which was introduced in Section 2.4

and discussed in the context of children in Section 3.4. Cognitive aspects of

Embodiment are further discussed in Section 4.1. These cognitive aspects of

Embodiment are based on the perspective of Embodiment that perception action

loops originate from the environment as information on invariants is derived from it.

Design aspects of Embodiment were derived from the cognitive aspects which were

then used to study Embodied intuitive interaction in children playing with toys.

2.2 EMBODIMENT IN INTERACTION DESIGN

Embodiment is the property of being manifest in, and a part of the world, and

of being grounded and situated in everyday rituals and activities (Dourish, 2001).

This state creates an engaged interaction with the world. Dourish’s concept of

Embodiment has been universally interpreted as an approach that leverages an

individual’s body movements, in interaction with physical objects and spaces, to

control computing systems.

In HCI design, the rise of tangible, haptic, ubiquitous and pervasive computing

has blurred the boundaries between bodily interaction and digital information.

Researchers such as Dourish (2001), Ishii (2007), Shaer & Hornecker (2010),

Sharlin, Watson, Kitamura, Kishino, & Itoh (2004) have referred to these

manipulations as ‘Embodied and tangible interactions’. Interaction design has


explored Embodiment in terms of interaction modalities such as graspable user

interfaces (Fitzmaurice, Ishii, & Buxton, 1995), frameworks and taxonomies to

design Embodied interactive products such as the framework for reality based

interactions (Jacob et al., 2008) and paradigms to design TEIs (Hornecker & Buur,

2006; Shaer & Jacob, 2009).

Shaer & Jacob (2009) proposed a specification paradigm for designing

Tangible Embodied and Embedded Interfaces (TEIs) that allows designers to specify

behaviour of TEIs without getting lost in the implementation details. TEIs offer an

explicit way of exploiting Embodiment through digital technology. Marshall, Price,

& Rogers (2003) discussed the role of TEIs in learning, and suggested that learning

through TEIs happens through two types of interactions: expressive and exploratory

(Marshall et al., 2003). Expressive interactions focus on external representations of

activities (Mellar & Bliss, 1994) such as model making and prototyping using clay,

or building using LEGO Mindstorms. Exploratory interactions, on the other hand, are

used to ‘explore’ a model that someone else has created, either by practical

manipulation or theoretical reflection. Chalmers (2001) argued that Dourish's

treatment of Embodied and tangible interaction with TEIs neglects expressive

interactions, and overemphasises exploratory interactions.

Although Marshall (2003) associated expressive and exploratory interactions

with TEIs, it is the physicality and materiality of the objects that allow these

interactions. Technology is a facilitator of these interactions, and is used to embed

and process digital information within objects. Mellar & Bliss (1994) explain

expressive behaviour in the context of learning by making models that are not

necessarily embedded with technology. Advances in technology now allow

computing elements to be embedded in objects so that the technology itself simply

disappears behind a façade. This has resulted in new interfaces, known as ‘embedded

interfaces’ (Gervais et al., 2016). These interfaces retain the richness and situated-

ness of physical interaction, while simultaneously embedding computing in existing

environments.

Material interaction and engagement with physical objects involves the mind,

body, and perceptual systems acting together (Malafouris, 2013). Traditional design

requires deep knowledge about, and sensibility for materials. The designer sees the

potential of this knowledge and sensibility, and uses it in the act of design. Design


then becomes a negotiation between form and function, and between aesthetics and

utility. The incorporation of computing elements in products (TEIs) offers an

additional dimension for these negotiations. Attempts are being made to re-imagine

computing as another material in a design toolkit in line with other materials such as

paper and cardboard. The result is the activation of the existing properties of

materials to create opportunities for richer interactions and experiences.

People demonstrate intuitive spatial relationships with objects, as the skill of

manipulation comes naturally to them (Zigelbaum, Kumpf, Vazquez, & Ishii, 2008).

Users determine the inherent functionality of objects from their physical and spatial

qualities such as shape, weight, size, and colour. Physical analogies and cultural

standards pertaining to the objects develop a sense of familiarity with the spatial

qualities of the objects. People use this familiarity to map spatial qualities to new

functions and tasks, resulting in new spatial mappings (Sharlin et al., 2004).

Actions performed on objects should be closely coupled to perception space

(i.e. the view and weight of the object), as this allows one to direct attention at one

place and time (Zigelbaum et al., 2008). For example, the action space of a computer

mouse is separate to its perception space (display screen) (Beaudouin-Lafon, 2000).

This results in the users having to divide their attention between the mouse and the

screen. In the physical world, humans use tools such as their hands to perform

actions on the objects in the same space and time (Dreyfus & Dreyfus, 2000). They

use their sensory cues and the condition and motion of the tools to track the progress

of their activity. The close coupling between action and perception space can be

achieved by unifying the input and output devices, for example, in tablets (Ullmer &

Ishii, 2000; Zigelbaum et al., 2008).

Thus, the concept of Embodiment has found its way through into the field of

interaction design. The primary objective has been to allow people to interact with

computing systems but maintain the traditional and natural interaction modalities to

allow users to engage with the systems. This study has investigated Embodiment

from the perspective of intuitive interaction design. The study has first investigated

Embodiment through the lens of cognitive science to determine design aspects of

Embodiment (discussed in Section 4.2). Experiment 1 then compared a physical

Jenga (Hasbro & Scott, 2001) and a virtual Jenga (Natural Motions & Scott, 2011) to

investigate intuitive interaction in children. The extent each of the design aspects of


Embodiment facilitates intuitive interaction in children playing with physical Jenga

and virtual Jenga was determined (Chapter 6). Experiment 2 investigated a physical

product and Tangible Embodied and Embedded Interface (TEI) to determine the

variability of intuitive interaction with respect to the design aspects of Embodiment

(Chapter 7).

2.3 INTUITION AND INTUITIVE INTERACTION

2.3.1 Intuition

The word ‘intuition’ has been liberally used in everyday life. Terms such as

‘gut feeling’, ‘hunch’, ‘sixth sense’, ‘second sight’, and so forth are interchangeably

used with intuition. People use these terms when they are unable to explain their

decisions, experiences, and actions. This happens because they have encountered

something similar in the past. Intuitive thinking is fast, unconscious, and often

automatic (Kahneman & Klein, 2009; Klein, 2003).This differs from rational

thinking that is slow, conscious, and deliberate. Research has suggested that most of

our thinking occurs outside of our consciousness (Bastick, 2003; Dreyfus & Dreyfus,

2000). Intuitive thinking is the cognitive process of using information previously

perceived by the senses (Bastick, 2003). This sensory information is used to make

insights, recognitions, and judgements (Harteis & Billett, 2013).

Intuition is a non-unitary construct with two systems—rational thinking and

intuitive thinking systems—underlying human thinking and reasoning (Myers, 2007;

Schon, 1982). Both the systems process the stimuli from the environment, ultimately

resulting in a decisive outcome. However, the two systems differ in the way the

stimuli are processed. Rational thinking is slow, deliberate, and conscious, and

makes the person feel as if s/he is in control of the decision making process. It

follows set rules and guidelines (heuristics) in coming up with a decision. Intuitive

thinking, on the other hand is fast, automatic, and non-conscious. It is capable of

learning by associating new stimuli with known stimuli in terms of causality,

contiguity in time and place, and shared characteristics. Intuitive thinking is capable

of reading the environment quickly, enabling a quick response in return.

Intuitive thinking constantly feeds suggestions to rational thinking that can

override these suggestions; however, rational thinking is often unaware that intuitive

thinking is influencing it. Intuitive thinking operates well in instantaneous situations,


and is less effective in long term planning. When intuitive thinking is met with a

situation that it cannot handle, it automatically invokes rational thinking; for

example, when an individual is met with a dangerous obstacle while walking,

rational thinking is triggered in order to analyse the danger. Thus, the decision

making process involves a continuous handover of controls between the two thinking

and reasoning systems; however, neither of the systems is aware of these handovers.

Intuition is a cognitive process, and builds upon past experiences (Bastick,

2003; Klein, 2003). People use their senses and past experience to recognise patterns

in a given situation. They then link these patterns to a series of past incidents

(Bastick, 2003). These traces of past incidents are derived from an individual’s

implicit memory (Schore, 2010). This probably is the reason people are not

conscious of their intuitive thinking, or of deriving knowledge from their past

experiences. However, Klein’s study was limited to experienced people. He

suggested that experienced people make intuitive decisions in real situations by

adding up experiences from various incidents (Zsambok & Klein, 1997).

Wickens, Gordon, Liu, & Lee (1998) used Rasmussen's (1993) Skill, Rule and

Knowledge- (SRK) based model of decision making to explain the use of experience

in intuitive decision making. They referred to the latter as ‘naturalistic decision

making’. Skill, Rule and Knowledge are the three levels of cognitive activity that

people operate in during task performance and decision making. The nature of the

task, and the level of experience with the task, decides the level of operation.

Highly experienced people operate at the skill-based level. Their performance

and decision making is at the subconscious level, and is often an automatic response

to a particular situation. People familiar with the task, but who do not have enough

experience, operate at the rule-based level. They look for cues or rules that they

recognise from past experience to make a decision. When the task is novel, and

people do not have any rules or cues to rely on from their past experiences, they

resort to analytical processing, using conceptual information. Problems are defined,

solutions generated, and the best course of action is determined before making a

decision. Thus, people operating at skill- and rule-based levels, people who are

highly experienced in a given task, and people who are familiar with the task but do

not have enough experience, are making decisions intuitively (Wickens et al., 1998,


p.198). On the other hand, people new to a given task could resort to non-intuitive

decision making.

Another explanation of how people use their past experience in intuition was

provided by Gore & Sadler-Smith (2011). They conceptualised intuition into

processes and mechanisms of intuiting, and the outcomes of these processes. They

suggested that although most of the research focussed on the outcomes of intuition, it

is the process of intuiting that uses past experience in thinking, reasoning, planning,

and decision making. The processes of intuiting are considered to be either domain-

general or domain-specific, depending on the similarity of the domain in which a

task is undertaken to the domain from which knowledge is derived. Domain-general

processes operate automatically across domains on the basis of the complexity,

uncertainty, and level of risk associated with a triggering stimulus. Domain-general

processes of intuiting include applying heuristics under uncertain conditions,

acquiring and activating domain-relevant schemas, and infusing affect into decision

making (Dane & Pratt, 2007). Domain-specific processes, on the other hand, are

specific to particular domains. They are activated autonomously based on learning,

schemas, and frequently encountered domains in which the task is located.

Intuitive judgements in information processing are a result of the application of

heuristics (Kahneman & Klein, 2009, p238). Slovic, Finucane, Peters, & MacGregor

(2007) suggested that the types of heuristics that contribute to intuitive judgements

are affective in nature. They empirically studied the role of affect in assessing

dangers and merits of hazards such as nuclear power. They found that people relied

on affect under time-pressured conditions. People were able to efficiently evaluate

risks and benefits using their ‘gut feel’ reactions. Intuiting involves drawing on an

affect pool of positive and negative affective tags associated with conscious or

unconscious representations (Bastick, 2003; Finucane, Alhakami, Slovic, & Johnson,

2000).

Domain-relevant schemas are activated under time-pressured situations to carry

out intuitive judgements (Klein, 2003). Physiological and emotional signals from the

body are perceived as warning or attention signals in intuitive decision-making

(Dunn et al., 2010). Domain-relevant schemas comprise procedural and declarative

knowledge acquired through learning and practice. Development of domain-relevant

schemas requires individuals to undergo repetitive practice under constant expert


feedback in controlled environments. Affect infusion is a process in which

“affectively loaded information exerts an influence on, and becomes incorporated

into, cognitive and judgmental processes” (Forgas, 1995, p. 101).

Intuiting or intuitive processes are fast, unconscious, characterised by

unconscious processing, and based on past experience (Blackler et al., 2010). In

terms of correctness of intuition, Blackler et al. (2010) suggested that the correctness

of intuition depends on the acquisition of experience relevant to the task. This means

that domain-specific processes have more chances of resulting in correct intuitions

than domain-general processes. Intuition and intuitive processes have been studied in

the field of interaction design to develop instruments, continua and frameworks for

the design and development of intuitive products (Blackler, 2008; Fischer, Itoh, &

Inagaki, 2015; Hespanhol & Tomitsch, 2015; Hurtienne, 2007; Macaranas et al.,

2015; Mihajlov, Law, & Springett, 2015). These frameworks, continua, and

instruments (discussed in Section 2.3.2) were studied to understand Embodiment in

the context of intuitive interaction for this study.

2.3.2 Intuitive Interaction

The term ‘intuitive interaction’ has been inconsistently used in Human Product

Interaction to characterise a product and its use. Ullrich & Diefenbach (2010)

suggested that a device can be made intuitive to use by designing it so that its

operation can be learned simply through its observation. This is possible through the

appropriate use of deliberate clues in the product that trigger past experiences and

prior knowledge. Ullrich & Diefenbach’s perspective on intuitive interaction is that

‘intuitiveness’ is an attribute of a product that drives an automatic and unconscious

use in individuals.

Intuitiveness is used as an attribute to describe user interfaces, and as an

assessment criterion for technical systems or user interface requirements. While

systems, products, and interfaces are referred to as intuitive, they are inanimate and

cannot undergo the processes of intuiting described in Section 2.3.1. Cognitive

psychology, and decision making and planning research have associated intuition

with human beings who are animate elements. Systems, products, and interfaces can

be designed to support people to interact intuitively, and this is what is referred to as

‘intuitive interaction’ and ‘intuitive interfaces’ (Antle, Corness, & Droumeva, 2009).


Early intuitive interaction research conducted by two research groups in

Australia and Germany focused on the types of experiential knowledge accessed by

people during intuitive interaction. They also discussed how products could be

designed for maximum intuitive interaction, and designed continua and models of

intuitive interaction. In several experiments, Blackler (2008) studied people using

complex products, and showed that they used knowledge gained from their

experiences with using other products to intuitively interact with interfaces. Using

the Technology Familiarity (TF) Questionnaire to determine a TF score for each

participant, she linked their prior experience to familiarity. She concluded that people

with higher TF scores complete tasks faster and more efficiently than those with

lower scores. Prior experience and familiarity with the elements of the tasks help

people to develop rules to perform given tasks (Rasmussen, 1993). They rely on

these rules to complete operations on the product. McEwan, Blackler, Johnson, &

Wyeth (2014) and O’Brien (2010) adapted the TF questionnaire and TF score and

subsequently used them in their study.

O’Brien (2010) studied interactions of younger adults and older adults with

everyday technologies such as an alarm clock, a Kindle (Amazon & Foxconn, 2007),

and a Flip camcorder (Pure Digital Technologies & Cisco Systems, 2006). Each

participant’s familiarity with each feature of the technology was evaluated using the

TF questionnaire and TF score. O’Brien found that prior knowledge of similar

technologies helped both younger and older adults to successfully interact with the

new technologies. However, there was a difference in performance levels between

younger and older adults across different technologies. Younger adults performed

better with the Flip camcorder in comparison to older adults, who had similar

technical experience but less frequent and recent experience with digital camcorders.

On the other hand, half of the older adults with low technology experience completed

each task on the Kindle, despite their low levels of technical knowledge and low

familiarity with other similar technologies. O’Brien suggested that older adults with

high technology experience could be using their recent experience with video players

to interact with features of the Flip camcorder. Thus, the user’s prior experience with

the technology, as well as their prior knowledge of similar (but not identical) devices,

should be understood.


McEwan et al. (2014) used the Game Technology Familiarity (GTF)

questionnaire to measure previous gaming experience of participants in a research

study that investigated the potential of different types of Naturally Mapped Control

Interfaces (NMCIs) for intuitive interaction in video games. He observed participants

playing racing video games with three types of NMCIs: directional, incomplete

tangible, and realistic tangible. Four types of GTF scores were evaluated, one each

for familiarity with the three types of NMCIs, and one related to previous play with

racing games. McEwan et al. (2014) found that naturally mapped devices provide

great potential for intuitive interaction for people with less gaming experience using

NMCIs, and/or high familiarity with real life activity. High familiarity gamers

demonstrated intuitive interactions with naturally mapped, as well as less naturally

mapped, controllers.

Blackler (2008) developed a continuum of intuitive interaction. It is shown in

Figure 2, bottom as it relates to the Intuitive Use of User Interfaces (IUUI) research

group’s continuum, shown in Figure 2, top. It includes factors that can be used to

facilitate intuitive interaction, from the simplest and most ubiquitous (physical

affordances) to the more complex (metaphors from another domain). It is universally

accepted in research and industry that products must be intuitive to use. However,

there is a lack of consensus in industry and marketing on what is meant by

intuitiveness, and how it should be measured. On the other hand, researchers do

agree that intuitive interaction is a subconscious application of prior knowledge that

leads to effective interaction (Blackler & Hurtienne, 2007). Blackler (2008)

operationalised conscious reasoning through reportability and verbalisability. People

interacting intuitively did not verbalise to report their actions and reasoning. Blackler

(2008) also used expectations, degree of confidence, latency, and evidence of past

experience and prior knowledge as measures of intuitive interaction.

Hurtienne (2009) and Naumann & Hurtienne (2010) investigated the

subjective consequences of intuitive interaction. They suggested effectiveness of

interactions, mental efficiency of the users, and user satisfaction as measures of

intuitive interaction. They developed a questionnaire, QUESI (Questionnaire for the

subjective consequences of intuitive use), that covers all the above measures of

subjective consequences. The questionnaire consists of 14 items, grouped into five

subscales: subjective mental workload, perceived achievement of goals, perceived


effort of learning, familiarity, and perceived error rate. The score of each subscale is

computed by evaluating the mean of all the responses to the items of that subscale.

The total score of the questionnaire is equal to the mean evaluated across all five

subscales. Hurtienne (2009) and Naumann & Hurtienne (2010) used QUESI to test a

number of products for intuitive interaction, including a game console, a music

player, a website, a mobile phone, and an operating system. They found that products

that users were familiar with scored higher QUESI scores than those that users were

not familiar with. However, they also found that people who were unfamiliar with

the product scored high QUESI scores when they attempted the questionnaire

immediately after using the product.

The German-based Intuitive Use of User Interfaces (IUUI) Research Group

presented a 'continuum of knowledge in intuitive interaction' (shown in Figure 2,

top), with types of experiential knowledge accessed during intuitive interaction based

on their frequency of cognitive encoding and retrieval (Hurtienne & Blessing, 2007).

Figure 2 Comparison of Intuitive Interaction Continua, shown through blue arrows (adapted from Blackler & Hurtienne, 2007)

Blackler and Hurtienne (2007) compared the two continua, and agreed upon

the forms of knowledge that are more accessible to some people than others. In the

Intuitive Use of User Interfaces (IUUI) continuum, the most basic knowledge

possessed by most people is innate knowledge. In Blackler’s continuum of intuitive

interaction, the simplest form of intuitive interaction is the use of physical

affordances that use Embodied knowledge of the world established early in life. The


physical properties of the artefact decide their possible interactions, such as grasping

and pulling. According to the IUUI’s continuum of knowledge in intuitive

interaction, such activities use sensorimotor knowledge; that is, knowledge derived

from Embodied interactions with the physical world, acquired very early in

childhood. For example, children learn about gravity, and build up concepts of speed,

motion, and kinematics. Image schemas also sit at the sensorimotor level.

The next most accessible form of intuitive interaction in Blackler’s continuum

is population stereotype; this is in line with the culture and sensorimotor levels in the

IUUI’s continuum of knowledge. It includes knowledge that is possessed by most,

but that is limited by societal bounds such as different meanings for hand and body

gestures, and different interpretations of linguistic structures in the same language.

Population stereotypical representations—such as a clockwise movement signifying

an increase—derive largely from the experience of cultural conventions. When

artefacts are designed in conformation with the population stereotypes, decisions are

made faster, and interactions are more likely to be correct and precise.

Expert knowledge is placed at the highest level in the IUUI’s continuum of

knowledge. However, it is slowest in terms of encoding and retrieval of knowledge,

and very few people have it. It is the knowledge held only by those adept at a

particular skill (such as the knowledge a user might apply to using a software

package such as Excel).

Blackler, Popovic, & Mahar (2010) suggested using familiar features from the

same domain (for example, file menus in software packages) to design intuitive

products that require the use of cultural and expert domain knowledge. However, if

suitable familiar features from the same domain are unavailable, designers might

have to use familiar features from another domain (e.g. a ubiquitous power symbol in

computers and mobile phones). Familiar features tend to be perceived affordances,

virtual objects such as icon buttons that invite pushing or clicking because, based on

prior experience with similar products, a user has learned that this is what it is for

(Johnson & Russo, 1981; Norman, 2013). Perceived affordances was, therefore,

placed on the continuum as being equivalent to familiar features (Blackler, 2008).

Finally, if the technology or context of use is completely new, designers can

leverage metaphors to communicate the intended interaction or to explain a

completely new concept or function. Metaphors are grounded in experience (Lakoff


& Johnson, 1980/2003); that is, analogies are retrieved from past experiences.

Elements of a known situation are mapped to a new situation; for example, concepts

of desktops can be applied to palmtops (Holyoak, 1991; Lakoff & Johnson,

1980/2003). Metaphors can apply across the IUUI continuum from sensorimotor to

expertise (for example, image schemas use metaphorical extensions). Thus, metaphor

in Blackler’s continuum is linked to IUUI’s sensorimotor, culture, and expertise

levels (see Figure 2). Across the sensorimotor, culture, and expertise levels of

knowledge, the IUUI continuum also highlights knowledge about tools.

The IUUI continuum of knowledge has an inherent dimensionality. The

frequency of encoding and retrieval of knowledge increases from the top to the

bottom of the continuum. Then, the further one moves towards the top level of the

continuum, the higher the degree of specialisation of knowledge, and the smaller the

potential number of users with this knowledge. Similarly, Blackler’s continuum has a

progression in ubiquity from left to right: The extreme left of the continuum

represents the most ubiquitous interfaces. These interfaces have features that people

have Embodied experience of; as a result, they can understand how to use them. The

next most ubiquitous interfaces are those designed with population stereotypes in

mind; those with familiar features which users might or might not recognise

depending on their pattern of past experience; and, finally, those using metaphor.

Ullrich & Diefenbach (2010) expressed their concern that, as a selling point in

product usability, the term ‘intuitiveness’ is characterised by features such as ‘easy to

learn’, and ‘natural interaction’. While this tendency has exposed numerous

components of intuitive interaction, it is still not clear what ‘intuitive interaction’

actually means. Ullrich & Diefenbach agreed with Blackler & Hurtienne’s (2007)

reference to intuitive interaction as the subconscious application of prior knowledge,

as it seperates intuitive interaction from usability. They also agreed with the

measures of intuitive interaction suggested by Blackler (2008) and Naumann &

Hurtienne (2010). However, they expressed their concern that the measures would be

more pronounced in familiar products, and less in innovative and new products. To

counter these concerns, Ullrich & Diefenbach (2010) approached intuitive interaction

as processes originating from ‘gut feeling’, rather than from ‘reason’. Accordingly,

they suggested subjective and experiential components of intuitive interaction: gut

feeling and magical experience as components of intuitive interaction. While ‘gut


feeling’ refers to the process of decision making, ‘magical experience’ refers to the

result of intuitive decision making. In addition to these two components, they agreed

with two parameters of intuitive interaction suggested in the literature: effortlessness,

and verbalisability (Blackler, 2008; Naumann & Hurtienne, 2010).

Intuitive Interaction research has continued to build on these ideas over the past

few years (Blackler & Popovic, 2015). Specifically, it has focussed on applying the

measurements, concepts, and parameters of intuitive interaction to other application

domains. Hespanhol & Tomitsch (2015) investigated strategies to design intuitive

interactive public spaces, and these have been referred to as a third wave of HCI.

Interactive spaces are an emerging field, and Blackler (2008) and Hurtienne,

Klöckner, Diefenbach, Nass, & Maier (2015) pointed out that metaphors and image

schemas that relate to something that is familiar to users would need to be applied to

innovative and inclusive products and technology that users are not familiar with.

Hespanhol & Tomitsch (2015) suggested that intuitiveness in interactive urban

spaces is mostly derived from the interactive experience that is the feedback from the

interactive systems. It is the perceived affordances of the systems that determines the

level of intuitiveness in urban environments. Hespanhol & Tomitsch thus presented

feedback strategies for intuitive interaction in urban interactive spaces: (i) directional

feedback versus scattered feedback; (ii) immediate and concrete feedback versus

delayed and abstract feedback; and (iii) visual feedback versus audio feedback.

Previous research mainly focussed on familiarity, prior experience, and

knowledge as facilitators of intuitive interaction. However, a focus on designing

products with familiar features could inhibit innovation. Different user groups have

different levels of familiarity with products and features, especially with regard to

technology products. Hurtienne et al. (2015) suggested that image schemas and

primary metaphors could bridge the gaps between inclusive, intuitive, and innovative

design. Image schemas are recurring, dynamic patterns of perceptual motor

interactions that give coherence and structure to users’ experience (Johnson, 1987, p.

xiv). Use of gestures or/and body interactions were also suggested as ways to design

innovative products that provide touchless natural interactions (O’hara, Harper,

Mentis, Sellen, & Taylor, 2013). The effectiveness and efficiency of the intuitive

mappings of gestures and body movements in product interactions depends on the

sensorimotor knowledge derived from everyday experiences (Kiverstein, 2010).


Mihajlov et al. (2015) found that simple touch gestures, such as a drag gesture,

were easily learned and retrieved by older people who had no prior experience with

touch-based interactions, but are familiar with a drag gesture from their real world

experience (e.g. the dragging of objects). However, in contrast, older people found a

rotate gesture difficult to use as it was not something that they used in their everyday

lives.

Macaranas, Antle, & Riecke (2015) highlighted the problems and

inconsistencies associated with gestural interactions across different users. Some

gestural and body interactions could lack physical or perceptual affordances and,

without very detailed instructions, this could make it difficult for users to know what

gestures and actions are supported. Macaranas, Antle, & Riecke (2015) compared

three interaction models for intuitiveness, each with a different approach to mapping

body actions to system controls: (i) metamorphic mappings, (ii) isomorphic

mappings, and (iii) conventional mappings. While, they did not find any statistical

differences between the three in terms of intuitive behaviour, participants

demonstrated different attention ratings for the three mappings. Those using

isomorphic mappings gave more attention to completing the task than to using the

system. Macaranas, Antle, & Riecke (2015) offered guidelines to explain which

application should be associated with each of the mappings.

Over the past few years, intuitive interaction research has continued to build on

ideas and concepts around ways to facilitate intuitive interaction (Blackler &

Popovic, 2016), and has focussed on various application domains (Desai et al., 2015,

2016; Swann, Popovic, Thompson, Blackler, & Kraal, 2015) and on elderly

populations (Lawry et al., 2011; O’Brien, 2010). However, there is limited research

that investigates children’s intuitive interaction. Israel, Hurtienne, Pohlmeyer, Mohs,

Kindsmüller, et al. (2009) studied aspects of intuitive interaction and classified these

aspects into effects, features, enablers, and facilitators. This classification offered a

perspective on how intuitive interfaces could be designed. They suggested image

schemas and affordances (among others) as enablers of intuitive interaction, and

noted physicality, familiarity, and Embodiment as the major facilitators. However,

they did not consider children as users in their experiments.

Brandenburg & Sachse (2012) did include children in their study investigating

the role of prior knowledge on intuitive interaction. The objective of their study was


to compare the intuitive behaviour of people with different levels of prior experience.

Children in this study represented user groups who had sensorimotor knowledge, but

no prior experience (referred to as ‘naïve users’ in the study). They tested a multi-

touch interface with children, adults without specific operating knowledge of

interacting with multi-touch interfaces, and adults with prior experience in using

smartphones and tablets. They asked participants to manipulate three objects shown

on the interface by using three gestures: cut, rotate, and scale. The object had to be

dragged into the middle of the screen, and a start button pressed, before executing the

manipulations. Two time durations were measured at the press of the start button:

Time to First Click (TFC), and Total Task Time (TTT). TFC captured the delay from

pressing the start button to the start of the gesture execution, and TTT measured the

total time from pressing the start button to completion of the gesture execution.

Brandenburg & Sachse (2012) found that the participants on the sensorimotor

level on Intuitive Use of User Interfaces (IUUI) continuum of knowledge (children)

were slower in terms of TFC than the participants at the culture level on IUUI’s

continuum of knowledge without knowledge of the tool (i.e. adults without prior

experience); the latter were, in turn, slower than the participants with knowledge of

the tool (i.e. adults with prior experience). They concluded that the results conform

with Hurtienne & Blessing's (2007) continuum of knowledge (refer back to Figure 2,

top). Brandenburg & Sachse (2012) also found significant differences in the

participant’s task performance (TTT) of the three gestures. All participants were

significantly slower in executing the cut gesture, compared to the scale and rotate

gestures. Scale was the second slowest gesture to perform, and rotate was the

slowest. Adults were faster in the execution of the cut and scale gestures than

children. When the participants were asked to repeat the same activity of

manipulating objects on the screen, their TTT did not improve equally over the three

gestures. While the cut and scale gestures improved in the second trial, there was no

improvement for the rotate gesture. However, Brandenburg & Sachse (2012) could

not conclusively determine the reasons for the differences.

The lack of adequate research that focuses on the role of sensorimotor

knowledge in children’s intuitive interaction was the motivation behind this research

study. Thus, its objective was to advance the existing and ongoing research

(Diefenbach & Ullrich, 2015; Hurtienne, 2009; McEwan, Blackler, Johnson, &


Wyeth, 2014; Mihajlov, Law, & Springett, 2015; Naumann et al., 2007) on intuitive

interaction, with specific reference to parameters and findings that can facilitate

children’s Embodied intuitive interaction. The continua of intuitive interaction

proposed by Blackler & Hurtienne (2007) were used to explain the results of the

study in Chapter 6 and Chapter 7, the outcomes of the study resulting in an Enhanced

Framework for Intuitive Interaction (EFII) explained in Section 8.1.2.

2.4 EMBODIED INTUITIVE INTERACTION

Intuitive interaction is strongly rooted in users’ Embodied skills. The

continuum of intuitive interaction (Blackler, 2008) and the continuum of knowledge

(Naumann et al., 2009) both highlight the role of Embodiment in intuitive

interaction. The simplest form of intuitive interaction relies on the Embodied

interactions with the world that use the sensorimotor knowledge acquired early in

childhood. This Embodied sensorimotor knowledge is acquired in everyday activities

and, by subconscious assimilation, becomes part of an individual’s behaviour

(Taraborelli & Mossio, 2008).

Embodied intuitive interaction is considered to be natural, and the simplest

form of interaction (Djajadiningrat, Wensveen, Frens, & Overbeeke, 2004;

Hurtienne & Israel, 2007; Sharlin et al., 2004; Terrenghi, Kirk, Sellen, & Izadi, 2007;

Zuckerman, Arida, & Resnick, 2005). However, there are very few studies that

investigated the role of Embodiment in intuitive interaction. These studies focussed

on Embodied metaphors that use body movements (Antle, Corness, & Droumeva,

2009) and image schemas as facilitators of intuitive interaction (Hurtienne & Israel,

2007). Metaphors allow retrieval of analogies from past experiences, and the

mapping of this retrieved information into the use of the new feature. Effectiveness

of the metaphors depends on how successfully they can be discovered and translated

into an appropriate action (Bakker et al., 2011). Thus, metaphors should be designed

in such a way that users are able to relate them to familiar things within or outside

the immediate context (Blackler et al., 2010).

Hurtienne (2009) investigated metaphors that represent recurring dynamic

patterns of bodily interactions—referred to as ‘image schemas”—derived from a

user’s experiential sensorimotor knowledge. Image schemas are “sensorimotor and

subconscious forms of knowledge representation” (Israel, Hurtienne, Pohlmeyer,


Mohs, Kindsmüller, et al., 2009). For example, the UP-DOWN schema was derived

from everyday experiences such as throwing a ball up in the air and climbing the

stairs. These schemas are deeply embedded in the human subconscious. Through a

series of experiments, Hurtienne (2009) showed that designing in accordance with

metaphorical extensions of image schema facilitates intuitive use. For example,

moving a slider up on a control increases the volume, and moving it down decreases

it. Hurtienne (2009) identified around 40 schemas based on human experiences that

can facilitate intuitive interaction. The image schema method is relatively simple and

easy to implement in design. However, interpretation of metaphorical extensions can

be culturally sensitive, and depends on domain-specific prior knowledge.

There is limited study in the field of Embodied intuitive interaction, which has

mostly focussed on Embodied metaphors and image schemas. To further advance

research in the field of Embodied intuitive interaction, this research study

investigated Embodiment through the lens of Embodied cognition to determine

design aspects of Embodiment (Section 4.2) which are then investigated through

experiments with children playing with a physical, virtual and TEI toys.

2.5 SUMMARY

This chapter has discussed the literature on Embodiment and intuitive

interaction. Perspectives of Embodiment based on how cognitive processes are

distributed between brain, body and environment were discussed. The perspective

used in this study suggests that cognitive processes in Embodiment rely on

perception action couplings which are initiated by the stimulus from the

environment. Cognitive aspects of Embodiment based on this perspective are

discussed in Section 4.1. Embodiment has also found its way into interaction design,

and its role in interaction design was, therefore, also discussed. Limited research in

Embodied intuitive interaction was discussed.

Intuition, which forms the basis of intuitive interaction, was discussed as a

cognitive process that builds on past experiences and prior knowledge. It is fast,

automatic, and non-conscious. Intuitive processes were explained using the Skill,

Rule and Knowledge- (SRK) based model of decision making, and Gore & Sadler-

Smith's (2011) concept of intuiting. The intuitive interaction research literature was

discussed, from the early research conducted by two research groups in Australia and


Germany (Blackler & Hurtienne, 2007), to recent work on intuitive interaction in

gaming (McEwan et al., 2014), interactive public spaces (Hespanhol & Tomitsch,

2015), and gestural interactions in older people (Martin Mihajlov et al., 2015). The

continuum of intuitive interaction and continuum of knowledge in intuitive

interaction were discussed.

Sensorimotor knowledge and Embodied experiences play an important role in

automatic cognitive processes. The relationship between Embodiment and intuitive

interaction has been addressed in the literature in the form of claims and assertions;

to date, however, this relationship has not been empirically investigated.

Chapter 3 discusses children’s Embodied intuitive interaction—the focus and

scope of this research study.


Chapter 3: Children’s Embodied Intuitive Interaction

Chapter 2 discussed the literature surrounding Embodiment and intuitive

interaction. The role of Embodiment in interaction design was discussed, ultimately

introducing the concept of Embodied intuitive interaction, and discussing the

relationship between Embodiment and intuitive interaction. This discussion

highlighted the limited research in this area, although intuitive claims of

Embodiment are made in the literature. There is even less research that investigates

the role of Embodiment in children’s intuitive interaction.

This chapter thus discusses Embodiment in children (Section 3.1). Specifically,

it highlights: the importance of epistemic and pragmatic interactions in Embodied

interactions, and the role of material and physical explorations in children’s

interactions; directly manipulated interfaces for children (Section 3.2); intuitions in

children (Section 3.3), including the role of intuitions in their learning; Embodied

intuitive interaction in the context of children as user groups (Section 3.4); and the

limited research on Embodied intuitive interaction in children.

3.1 EMBODIMENT IN CHILDREN

There is abundant literature on Embodied cognition, and researchers have

successfully applied ideas from Embodied cognition in interaction design (some of

which were discussed in Section 2.2). Embodiment has departed from a traditional

belief that cognition is completely situated in the brain. On the contrary,

Embodiment believes in a distributed theory, that is, that cognition is shaped by an

individual’s physical interaction in the world. As discussed in Chapter 2,

Embodiment highlights the role of the human body, brain, and environment acting

together in complex physical, social, and cultural contexts to determine cognitive

structures.

However, the question that has motivated this research study is how we might

use ideas about Embodiment in interaction design for children. The concept of

Embodiment has found its way into HCI; however, it has been underutilised as it has

been limited only to action-based or body interactions, and has mainly focussed on


technology capabilities. This is even more the case when it comes to interaction

design for children. Research in interaction design for children has usually focussed

around play and learning and, in this context, the goal has usually been to provide

engaging experiences.

The role of action and environment in children’s development, learning, and

play is evidenced in the literature. Knowledge acquisition in children is a continuous

process of invention and re-invention as they interact with the surrounding world in

their developmental stages (Piaget, 1952). According to Piaget’s theory of Genetic

Epistemology, children acquire three types of knowledge through their actions and

activities in the world, and this acquisition is hierarchical in nature. Physical

knowledge, the first and most basic knowledge acquired by children, is the

knowledge about objects in the world that people derive through their perceptual

properties. This is followed by abstract knowledge, knowledge that is learnt and

invented by children. Finally, social-arbitrary knowledge pertains to a specific

culture, and is learnt from people within that cultural group. Acquisition of social-

arbitrary knowledge requires the prior acquisition of both physical and abstract

knowledge. Furthermore, no knowledge can be acquired without the prior acquisition

of physical knowledge.

Intelligence develops as cognitive structures are formed from patterns of

physical or mental actions. Piaget (1952) rejected the idea that development of

knowledge and intelligence in children is genetically programmed in the brain;

rather, he believed that cognitive structures are formed in stages as children grow in

age. Piaget described development in children as a linear progression through

discrete stages of reasoning that correspond roughly with children’s ages: sensory-

motor, pre-operation, concrete operation, and formal operation stages.

Genetic epistemology is often conceptualised through the lens of Embodiment

because it emphasises the emergence of cognitive abilities grounded in sensory-

motor abilities. It also emphasises the role of the physical world in the development

of intelligence and knowledge. However, the age/stage aspect of Piaget’s theory does

not adequately consider individual differences in the development of intelligence that

result from each individual’s unique interactions with the physical world. In contrast,

Embodiment shifts the focus away from development as a linear progression


culminating in the development of abstract reasoning abilities, and towards a situated

and integrated view of development.

Thelen & Smith (1996) discussed how walking depends on the child-

environment relationship and on other variables, such as limb weight and size. They

argued that the environment-dependent complexity of learning to walk rules out a

programmed linear progression in developmental stages in children. Children and

infants do not have a pre-set goal to learn to walk. The environment needs to provide

for the child’s learning. Variations in the environment ensure that people generate

different movements in different environmental situations (Bernstein, Latash, &

Turvey, 1996). Different circumstances in an environment result in different patterns

of muscle excitations for limb and body movements. Thelen and Smith (1996)

explained the role of variations in the environment in children’s walking, kicking,

and stepping. Limbs are dynamic systems whose properties (functional and physical)

depend on their interactions with body and environment. Steps are self-organising

motions that emerge from the history of changes to the environment system.

Alternate stepping (walking), then, emerges from the child-environment interactions.

The body and its movements play an important role in processing old ideas, as

well as generating new ideas. This results in better retention of knowledge. Cook,

Mitchell, & Goldin-Meadow (2008) observed children, encouraging them to make

gestures while learning a new arithmetical strategy. They found that children who

were told to gesture during learning retained 85% of their post-test gains four weeks

later, compared to only 33% of children who were told to only speak during learning.

Children’s increased use of technology in play and learning means that they are

spending less time interacting with the environment, performing activities that trigger

sensory-perceptual structures. These interactions and activities help develop

knowledge structures that aid children’s cognitive development (Livingstone, Marsh,

Plowman, Ottovordemgentschenfelde, & Fletcher-Watson, 2015). Thus, an

Embodied perspective to design could focus on the fact that children create meaning

through action (body movements and manipulation). This, in turn, could inform ways

to design children’s products, physical products, and TEIs that are situated in activity

and in the world.

Children’s Embodied cognitive processes are not different to adults’ Embodied

cognitive processes. Thus, the literature that was discussed in Chapter 2 also applies


to children. However, these processes develop differently in children, depending on

their social and physical environment and age. Nunnally & Lemond (1974)

suggested that children develop Embodied cognitive processes through explorations

of artefacts and the environment. This study considered that these explorations are

distinctly different to those in adults, and this is evident in the ways children

approach solving new challenges and problems (Kail, 2012).

Embodiment in children has formed the basis for this study on Embodied

intuitive interaction in children. Cognitive aspects of Embodiment are further

discussed in Section 4.1, and although not discussed in literature in the context of

children, these aspects do apply to children as well.

3.1.1 Children’s interactions

Children obtain and utilise information about persistent properties of the

environment (Gibson, 1988). Their use of object manipulation and environment in

development, play, and learning has been discussed in the literature (Kail, 2012; Ruff

& Saltarelli, 1993). Children learn and develop new skills and knowledge about the

environment through object manipulations and interactions with artefacts in the

environment that come their way (Jones & Lederman, 2006). McCall (1974), in one

of the first investigations of exploratory manipulation in human infants, suggested

that early exploration in children is an investigation of the raw sensory perceptual

feedback of objects. It does not require children to expend a significant amount of

energy to manipulate the objects in an environment. Visual and tactile inspection of

objects is accompanied by motor schemes such as shaking, banging, turning the

object over and over, and shifting it from hand to hand. Gibson (1988) suggested

that, in perception, these motor mechanisms are an essential part of seeking

information.

These manipulations with objects are not random; rather, they are driven by

intentions, curiosity, and intrinsic motivations (Bruner, Jolly, & Sylva, 1976). As

children grow older, their interactions become more relational and symbolic, and this

is evident in their imaginary play with objects. In turn, this suggests the emergence

of new cognitive abilities from these nonspecific manipulations. The acquisition of

new knowledge, age-related capacities, and changes in environment, all contribute to

the development of cognitive abilities.


Antle (2011) suggested that children simplify problem-solving tasks by

manipulating the environment (in ways such as physically turning jig-saw pieces to

see where their shape fits into the puzzle), and stated that children ‘think with the

hands’. Kirsh (2013) deconstructed such activities using concepts of pragmatic and

epistemic actions. Epistemic actions, such as search and planning, alter the world to

aid and augment the cognitive processes; for example, arranging furniture in a room

to decide on the layout, or adjusting alignments and placements of objects to obtain

an ideal fit to deflect balls on the targets in the game of Osmo (Tangible Play,

2014b). Pragmatic actions, on the other hand, are actions that alter the world when a

physical change is required for its own sake; for example, actually placing a puzzle

piece into the scene it is helping to create.

While epistemic actions could fail to assist in reaching a goal, they could

reveal completely unexpected information and shortcuts that would have been very

difficult to find by following a straightforward approach (Feinstein & Meshoulam,

2013). While such interactions might take longer, they are effective in offering

appropriate directions for children to reach a goal. Thus, an effective tool for

children’s learning and play should both facilitate goal-oriented pragmatic activity,

and provide a balanced amount of epistemic task space (Kirsh & Maglio, 1994).

3.1.2 Role of materiality in children’s interactions

Directly manipulated interfaces allow children to explore an artefact and, in the

process, they learn its use. This behaviour of exploring and learning comes naturally

to children. Montessori (2013) described the use of materials and activities to help

children develop their sensory capabilities. The materials put children in control of

their learning process, enabling them to learn through personal investigation and

exploration.

Children build and experiment with manipulative materials and, in the process,

develop a rich understanding of, and think about complex and abstract concepts.

However, Formal methods which involve manipulation of symbols are used to teach

complex concepts especially those related to dynamics and systems. As a result,

students are able to grasp and understand these concepts only when they grow older

and reach higher grades (year levels) at school, when they have developed more

mathematical expertise (Gopnik, Meltzoff, & Kuhl, 2009). Montessori (1914/2011)

introduced new ways of teaching children using materials and activities which tap on


children’s sensory capabilities. These materials allow children to explore and

investigate through physical and material manipulations and in the process be in

control of their own learning. Resnick et al. (1998) used the Montessori method to

develop traditional toys with computing elements embedded within them. Such

digital manipulatives enable children to explore a new set of concepts which would

otherwise be considered too advanced for them.

Children’s use of material and spatial properties in their interactions with toys

is one of the criteria used to investigate Embodiment in children playing with

physical, virtual and TEI toys ( Chapter 6 and Chapter 7 ). Spatial and material

properties of artefacts as a design aspect of Embodiment is discussed in Chapter 4.

3.2 DIRECTLY MANIPULATED INTERFACES FOR CHILDREN

The advantages presented by children’s material and physical interactions have

been the focus of research in TEIs for children, especially directly manipulated ones.

Researchers are interested in designing products that offer the advantages of physical

and material interaction. At the same time, however, the products need to incorporate

new innovations in technology.

Material and physical manipulations enable children to describe the actions

available in a physically shared space (Rogers, 2011). Fernaeus & Jacobsson (2009)

investigated the role of materiality in clothing in human culture. They concluded that

material properties of clothes allow people to present themselves to others through

their surface appearance. When people come in contact with the clothes, the

materiality of the clothes provide convenience, comfort, and warmth. Materiality also

serves a range of communicative functions; for example, it can suggest appropriate

group behaviour, group belonging, and expected interactions (Entwistle, 2015).

Taking inspiration from theatrical costumes, Fernaeus & Jacobsson (2009) suggested

that artefacts could be designed with material properties that conveyed information to

the user. This could then indicate to the user an artefact’s modes of operation and, in

turn, the behaviours and interactions that could be expected at a certain point in time

during interaction with the artefact. Fernaeus & Jacobsson (2009) used this concept

in developing a dinosaur toy that allowed children to decide its operation, using

accessories such as stickers, badges, bracelets, and decorative costumes. A dinosaur


toy, when decorated with a bracelet, was set in a play mode; when it wore pyjamas, it

went into sleep mode.

Seo, Arita, Chu, Quek, & Aldriedge (2015) developed Stampies to investigate

how young children associate meanings that allow playful physical interactions with

materiality. They were interested in understanding if the associations could be

extrapolated to digital interactions. Stampies consisted of physical objects made out

of different materials (wood, felt, silicone, and plastic), with a unique pattern of

conductive thread underneath. The Stampies were used with an iPad drawing

application that consisted of colour bands and graphic images of animals, fruits,

musical instruments, and clothing. These graphic images were both coloured and line

drawn. Children were asked to choose a Stampie to associate with each of the graphic

images on the ipad app. Then, they were asked to recall (on a blank canvas) what

each Stampie represented. Children were finally asked to create an artwork using the

chosen Stampies. This process was carried out separately with colours, coloured

images, and line-drawn images. The findings suggested that children associated

materials with meanings. Children associated certain materials with specific objects,

for example, wood with musical instruments. However, a material’s feel (whether

soft, hard, etc.) dominated as a guiding factor when reassessing the associations; for

example, children mostly preferred soft materials such as felt and silicone,

irrespective of their graphic image.

Children are increasingly using TEIs in gaming (Crowle, Boniface, Poussard,

& Asteriadis, 2014), patient rehabilitation (Vogiatzaki, Gravezas, & Solutions,

2013), collaborative coordination in time-critical situations (Fischer et al., 2014), and

in education (Gardner & Elliott, 2014). Efforts to develop TEIs for children are more

prevalent in education and (to some extent) gaming, than in any other field. In one of

the first examples of attempts to encourage social interactions in children while

playing digital games, Brederode, Markopoulos, Gielen, Vermeeren, & de Ridder

(2005) developed ‘pOwerball’. The objective was to bring together children, with

and without learning disabilities, to play a game. The game consisted of a tabletop

with graphic virtual elements which could be controlled through manipulation of

tangible objects on the tabletop. TEIs could pose challenges for children as they

could be interacting with multiple interfaces at the same time. These interfaces could

be physical or virtual, or both, thus requiring knowledge from physical and virtual


worlds. Knowledge from the physical world mostly relates to day to day experiences,

and are Embodied in everyday activities. On the other hand, knowledge from the

virtual world relates to experience gained from using virtual elements that are

design-dependent. The tendency in the past has been to look at TEIs through the lens

of pervasiveness (Fischer et al., 2014; Gardner & Elliott, 2014; Ricci, Piunti,

Tummolini, & Castelfranchi, 2015). Thus, the focus has been more on the technical,

rather than the human-centred, aspects of design.

Direct interaction with, and manipulation of objects helps children to ascribe

meaning to them. As technology becomes more accessible to children, new interfaces

and systems should be designed to take advantage of material and physical, as well

as technological, interactions. Tactile direct interactions were used as an interaction

modality in this study. Direct interactions with toys were coded for Embodiment and

intuitive interaction, discussed in Chapter 6 and Chapter 7

3.3 INTUITION IN CHILDREN

The interest in children’s intuition appears mainly in the fields of child

development and the learning sciences, fields that are mainly interested in children

educational needs. Schon (1982) suggested that children develop intuitive

understandings very early in childhood which enable them to give correct answers to

abstract questions. However, when they start attending school, teaching interrupts

their intuitive thinking through the introduction of models and procedures that

interfere with their intuitions; furthermore, as their intuitions are overridden, they

make mistakes. Later on, when children reach higher grades (year levels) at school,

they achieve a more developed, schooled intuitive understanding of various concepts

that enables them to answer correctly once again.

Choi (1993) argued that children are able to answer correctly again because

they are able to make higher order intuitive connections and understandings, and

with increased expertise levels. The development of expert skills leads to the

development of intuitions that Choi referred to as ‘Matured Intuitions’. However, this

takes years of learning, and Fischbein (1999) suggested that children either give up

on learning some of the concepts, or grow up with an incomplete and inaccurate

understanding of them.


Fischbein (1999), Noddings and Shore (1984), and Resnick (1986) proposed

that teaching models should support and enhance intuitive thinking in children, and

should not contradict the knowledge that is inherent in children when they start

school (Fischbein, 1999; Noddings & Shore, 1984; L. B. Resnick, 1986). Early

intuitions about numbers should form the basis of the elementary school mathematics

curriculum. These would then need to be extended to support secondary school

mathematics (Resnick, 1986). Resnick explained how these intuitions might function

in the child’s construction of new knowledge. Human beings acquire information and

convert it into well-structured, self-consistent, action-oriented representations of

reality, using language, logic, reasoning, and tools and instruments. This knowledge

through reasoning becomes an autonomous activity.

Intuitions are shaped by experiences, and they can be used as the basis for

understanding abstract mathematical concepts (Fischbein, 1987). Children develop

an understanding of space, geography, the cultural environment, and practices related

to professions from the daily experiences that shape their intuitions. Fischbein

further added that intuitive knowledge is immediate and self-evident, and that this

helps children to easily understand abstract concepts. Thus, teaching and learning

models should build on children’s intuitions.

Children readily relate to problems that have origins in the real world around

them. Zuckerman et al. (2005) explored the role of physical materials in building

children’s existing intuitions, and in developing their mathematical and scientific

intuitions. Clement (1994) suggested the use of physical materials to model

mathematical situations. Physical materials can be extremely useful in enabling

children to construct an essential intuitional foundation for mathematical thinking

and problem solving. For this level of intuition to develop, children should be

familiar with the physical materials from their everyday use in learning, or from

other domains (Blackler et al., 2010) in which they construct symbolic meanings and

operational definitions (O’Brien, 2010).

Disessa (1988) considered the role of familiarity in children’s intuitive

thinking. When children are confronted with unfamiliar questions, they are unable to

make connections with things around them, and are thus unable to use their intuition.

They make connections with their experiences and with objects around them based

on the clues that they pick up from the questions; they then choose appropriate


intuitions based on these connections (Disessa, 1988). To make these connections,

children should be familiar with clues, such as the symbols and language used in the

questions. This is very important, especially when children are presented with

abstract concepts and principles such as gravity and infinity.

To summarise, there is some evidence that intuitive learning in children results

in a less stressful learning experience, especially with the learning of abstract

concepts that deal with formally based types of certainty (Fischbein, 1987). Attempts

must be made to facilitate intuition-based learning (Fischbein, 1987). One of the

ways to do this is to allow children to learn in the real world and environment with

real objects (Clement, 1994), and allow them to apply their existing knowledge.

Intuition and intuitive processes have been studied in the field of interaction

design to develop instruments, continua and frameworks for the design and

development of intuitive products (Blackler, 2008; Hurtienne & Israel, 2007).

However, there are limited studies that investigate intuitive interaction in children

(Brandenburg & Sachse, 2012). This study has thus furthered the research on

intuitive interaction in the context of designing children’s products. Intuitive

processes in children were used to understand intuitive interaction and code these

processes in Experiment 1 and Experiment 2, discussed in Chapter 6 and Chapter 7.

3.4 CHILDREN’S EMBODIED INTUITIVE INTERACTION

Research suggests that Embodiment offers a natural or intuitive form of

interaction (e.g. Rosenbaum, Eastmond, & Mellis (2010). Antle et al. (2009a)

investigated these claims in the context of HCI. They suggested that claims of a

natural or intuitive form of interaction could be due to Embodiment being associated

with the human body and its participation in the environment. For example, humans

use their Embodied experiences to understand abstract concepts by using metaphors.

Antle et al. (2009b) developed a responsive auditory environment where Embodied

metaphors were used to map physical body movements, resulting in sound output

with percussive sounds. They used orientational and ontological metaphors to

represent parameters of music such as volume and pitch. They carried out empirical

experiments with adults (18–40 years old) and children (7–10 years old) interacting

with the auditory environment, with and without Embodied metaphors. Participants

performed specific physical movements to vary one of the sound parameters


(volume, tempo, pitch, and rhythm) to produce different sound sequences. They used

the following measures to determine whether the systems were intuitive to use: (i)

time taken to practice creating a sequence; (ii) accuracy of the final presentation of

the sound sequences; (iii) accuracy of the verbal explanation for each sequence task;

(iii) Intrinsic Motivation Inventory (IMI); subscales for enjoyment and interest; (iv)

perceived competence; and (v) individual statements related to ease of learning,

intuitiveness of learning, and amount of concentration required to learn. The results

revealed that the Embodied metaphor-based system was more intuitive than the

system without the Embodied metaphors. The experiments proved the effectiveness

of Embodied metaphors in intuitive interaction.

Completion of even a simple task involves a complex web of mental and

physical manipulations. Image schemas are translated into physical variables that are

then mapped into digital variables for interaction with technology. However, there

are multiple cross-mappings among these variables. As Embodied metaphors are

derived from Embodied experiences, the effectiveness of Embodied metaphors with

children could be questionable. This is because children’s Embodied experiences

could be more limited than those of adults, and are largely varied across

demographics and cultural backgrounds (Eriksen, 2001; Jenkins, 2014). In the Sound

Maker developed by Antle, Droumeva, & Corness (2008), children manipulated

pitch, volume, and tempo of sound through whole body interaction. However, some

children were unable to discover the mapping of the system within the set timeframe.

For this reason, Bakker, Van Den Hoven & Antle (2011) developed Moving Sounds

(MoSo) to study how tangibles can support children’s learning of abstract sound

concepts such as pitch, volume, and tempo. Bakker, Van Den Hoven, & Antle (2011)

identified Embodied metaphors that were used unconsciously by children aged 7-9

years to represent these musical sound concepts. Directly manipulated products were

developed to represent each of the identified Embodied metaphors. For example, a

puller artefact was used to represent the near-far Embodied metaphor, with ‘near’

representing low pitch, and ‘far’ representing high pitch.

The implementation of Embodied metaphors in the directly manipulated

products was evaluated in a user study with 50 children aged 7–9 years. The children

were asked to listen to a short sound sample in which the concept changed from one

extreme to another, for example, soft to loud volume. Each child was then given an


Embodied metaphor-based product to enact the change of sound. Children then

exchanged the products, and the exercise was repeated until all children had played

with all the products. The entire exercise was video recorded for analysis. Bakker,

Van Den Hoven, & Antle (2011) found that there were consistent patterns of

interactions with some products, while the interactions with others were inconsistent.

Almost all children used rotating artefacts to enact changing volume and the

Embodied schema slow-fast. On the other hand, children moved a stick with beads

attached to it in different ways to enact changing volume. None of the movements

resulted in low-high schema. At the same time, most of the children working with

pitch moved this artefact low-high. This means that some Embodied metaphors such

as low-high might be less appropriate for physical artefacts for change in volume, but

more appropriate for change in pitch.

Bakker, Van Den Hoven, & Antle (2011) pointed out that the effectiveness of

the Embodied metaphors in full body interaction systems such as the one described

in Antle et al. (2009b) depends on how successfully children are able to discover the

metaphors and translate them into an appropriate physical action. Directly

manipulated interfaces can exhibit features that children can easily discover and

activate.

Research in Embodied intuitive interaction has mainly focussed on

metaphorical interactions as facilitators of intuitive interaction. However,

Embodiment goes beyond body movements. Researchers in cognitive science have

identified aspects of Embodiment (as discussed in Section 4.1). However, the

relevance of these aspects to design has not been previously discussed, but is now

explored in this study (in Section 4.2). Physical products and TEIs are anticipated to

provide benefits to children in terms of usability, user experience, and learning and

development. However, there is limited empirical evidence to validate these claims.

Research in Embodied interaction for children has predominantly focussed on the

theoretical evaluation of tangibles (Antle, 2007); on the technology needed to embed

electronics in physical objects (Olson, Atrash Leong, Wilensky, & Horn, 2011); and

on the research methods to evaluate the effectiveness of tangibles for children

(Zaman et al., 2009). This research study focussed on the interactions with directly

manipulated interfaces to investigate children’s Embodied intuitive interaction.


There is limited study in the field of Embodied intuitive interaction (Antle et

al., 2009a; Hurtienne & Israel, 2007), which has mostly focussed on Embodied

metaphors and image schemas. Although limited studies on intuitive interaction in

children associate children with sensori-motor knowledge (Brandenburg & Sachse,

2012), there is a lack of research that focusses on Embodiment and intuitive

interaction in children. To further advance research in the field of Embodied intuitive

interaction, this research study investigated Embodiment through the lens of

Embodied cognition to determine design aspects of Embodiment (Section 4.2).

3.5 SUMMARY

This chapter has discussed the literature related to children’s Embodied

intuitive interaction, and highlighted the importance of their epistemic and pragmatic

activities. Epistemic interactions could prolong the time taken to reach a goal. At

times, they could even fail to assist in reaching a goal; however, they could also

result in unexpected discoveries that could make the journey to the goal easier.

Children find meaning in their interactions with material and physical

properties of objects. If the material and physical meanings derived through

perceptual sensory systems could be extrapolated to digital interactions, it could

result in interfaces that take advantage of both material and physical properties and

technological advances. Children unconsciously relate complex abstract concepts

with physical and material interactions. However, based on their past experience and

previous knowledge, they associate certain concepts with specific movements or

interactions. Thus, use of Embodied metaphors in tangible physical interactions for

intuitive use is suggested in research. However, considering the advantages offered

by material and physical properties of objects in interactions, there is scope for the

study of Embodied intuitive interaction in directly manipulated interfaces. This

research study has thus focussed on children’s Embodied intuitive interaction in the

context of directly manipulated interfaces.

Chapter 4 discusses cognitive aspects of Embodiment, based on the perspective

that suggests that the perception action loop originates from the environment in the

form of stimulus that is sensed by children’s perceptual sensory systems. Design

aspects of Embodiment derived from these cognitive aspects are discussed. These


design aspects were used in the empirical investigation of Embodied intuitive

interaction in children.


Chapter 4: Aspects of Embodiment

Chapter 2: discussed theories and frameworks pertaining to Embodiment

through the lens of cognitive science. Chapter 3 discussed Embodiment in the

context of children. The perspective of Embodiment used in this study is one that

suggests that cognitive processes are distributed between body, brain and

environment through continuous perception action loops and it is the environment

that initiates the process of perception and action loop and not the brain. In other

words, it is the environment that provides stimulus to trigger perceptual sensory

systems in children. This is the meaning of Embodiment that Anderson (2003),

Wilson & Golonka (2013) and Wilson (2002) have discussed in the context of

cognition. Thelen (2008) and Payne (1991) takes the same approach to describe

development of abilities and skills in humans and interactions with displays

respectively.

Section 4.1 discusses cognitive aspects of Embodiment. These aspects are

representative of perspective of Embodiment presented by Anderson (2003), Wilson

& Golonka (2013) , Wilson (2002) and to some extent Thelen (2008) in cognitive

science and perspectives from research in interaction design presented by Shaer &

Hornecker (2010) and Payne (1991). Section 4.2 presents design aspects of

Embodiment as a framework for Embodiment, derived from the cognitive aspects of

Embodiment. These design aspects were used to address the research sub-questions

in Experiment 1 and Experiment 2.

4.1 COGNITIVE ASPECTS OF EMBODIMENT

Physical grounding is the central, defining characteristic of Embodiment

(Anderson, 2003). Abstract representations acquire meaning from the real world

through physical grounding, and involve cognitive contents grounded in terms of the

agent’s Embodied experience and physical characteristics (Harnad, 2003; Lakoff &

Johnson, 1999; Vogt, 2002).

The traditional approach to cognition lacks the resources to ground its

representations (Harnad, 2003). Vogt (2002) further explains that Cartesianism,

Brainbound, and GOFAI (Section 2.1) rely entirely on human interpretation to give


meaning to symbols. Therefore, they implicitly require the human grounding

capacity to serve as an intermediary between its outputs and real-world activity.

Physical grounding of ‘chair’ requires specific physical skills and experiences related

to sitting and other related activities such as standing, walking, and running

(Anderson, 2003). Grounding requires people to have experienced an act of sitting,

and to have knowledge of chairs. This allows people to perceive the scene in front of

them, and to determine if it affords sitting. Experience, skill, and knowledge in using

chairs also enable people to understand other concepts such as ‘tables’. These

concepts are also related to semantics (Barsalou, Santos, Simmons, & Wilson, 2008)

and to context of use (Maguire, 2001). In other words, Embodiment and physical

grounding require a consideration of the social significance of objects and context.

The cognitive aspects of Embodiment that form the basis of physical grounding are:

Real-world and real-time aspect, Evolutionary aspect, Cognitive offloading, and its

Social aspect. Each of these aspects are discussed below.

4.1.1 Real-world and real-time aspect

People carry out cognitive activities in the real world, with real people and objects,

and in real spaces. The real-world activity is possible because cognition involves

perception and action (Clark, 2008; Eelen, Dewitte, & Warlop, 2013; Goodwin,

2000). Knowledge is situated in activity, and in social, cultural, and physical contexts

(Brown, Collins, & Duguid, 1989). However, this knowledge requires direct

interaction with the things that are to be cognitively processed (Clark, 2013).

Driving, communicating, and arranging puzzle pieces (i.e. trying to decide where

they fit into the whole picture) are some of the examples of such activities. Situated

activities are possible only in the real world and in real time, where information

perceived through the senses affects the way a particular task is executed; this

execution, in turn, affects the environment. Some situated activities might require

mental activities, such as the communication of past information, to perform situated

activities (Leakey, 2008); for example, the use of language in daily activities.

Children are considered as dynamic systems (Forrester, 1995; Thelen & Smith,

2006) as they build a dynamic history of activities in the real world. The real-world,

real-time nature of these activities gives momentum to behavioural acts, so that the

system is always impacted differently by its activities. Thelen & Smith (1996)

viewed development in children as a series of behavioural patterns evolving over


time, with time periods that are stable in terms of behaviour. Each behavioural act

occurs over time, showing courses of activation, peak, and decay. Every act changes

the system in some way or the other, such as properties, structural and so on. These

system changes build a history of acts over time. Thus, repeating the same behaviour

over time stabilises the system at some point resulting in habituation or learning. The

real-time, real-world nature of behaving/acting can result in different behavioural

outcomes under similar conditions. The outcomes depend on the immediate previous

history of the system.

Interaction in the real world is associated with response and feedback in real

time. This could result in individuals having to perform cognitive activities under

pressure. Certain activities such as flying an aircraft, and situations such as solving a

puzzle within a certain time limit, require fast responses to the environmental

stimulus. These responses need to evolve and adapt continuously to the ever-

changing environment. Responses in such situations cannot be derived from a mental

representation of the environment, as there is no time to build the representations of

the ever-changing environment. Cartesianism-, Brainbound-, and GOFAI-based

systems could either become stuck in a ‘confused state’, or generate an inaccurate

response. The systems might not have the representation for that particular situation

in the environment. Kirsh (2013) refers to this situation as the ‘representational

bottleneck’.

Embodied cognition, on the other hand, suggests that people generate situation-

appropriate responses on the fly to avoid representational-bottleneck. Individuals

tend to step back, observe, assess, plan, and then take action in the absence of time

pressure. However, they behave differently under time pressure, and this could lead

to a different outcome to the problem-solving task.

4.1.2 Evolutionary Aspect

Complex behaviour in organisms is linked to evolutionary theory (Hayles, 2010).

Most animals have significant built-in behavioural expertise, without having to

explicitly learn it from scratch. This expertise is developed as an organism evolves

over many years. Organisms possess specific physical features and behavioural

characteristics for coping with a specific environment (Anderson, 2005). As

organisms evolve, their physical and behavioural features undergo changes. The

cognitive adaptations then develop in light of the organism’s physical or structural


features and these, in turn, change in the context of reliable environmental features.

This is known as ‘emergent behaviour’ in organisms.

The central idea of the evolutionary aspect of Embodiment is the re-utilisation

of existing behaviours to solve new problems (Anderson, 2005). Clark (2005),

Turner (2013) and many others have used crickets as an example to explain the

evolutionary aspect of Embodiment. Cricket chirps are low in amplitude, and this

makes it hard to reach prospective mates. So, they build a specially shaped burrow

consisting of a hollow bulb underground, connected by a narrow constriction to a

flared tunnel opening into the air. The cricket chirps from the intersection of bulb and

horn, and adjusts the burrow until it experiences the right resonant frequency. Turner

(2013) suggested that the original purpose of the burrow must have been to protect

themselves from predators, rather than to attract mates, and that this behaviour is an

evolutionary development. Similar re-utilisation of physical structures and

behaviours should be expected in cognition (Clark, 2013).

Abstract reasoning capabilities can be traced back to perceptual and motor

inference in primitive creatures through metaphorical mappings across the relevant

domains (George Lakoff & Johnson, 1999, p4). They emphasise that the neural and

cognitive mechanisms that help organisms to perceive and move around also create

conceptual systems and modes of reasoning. Thus, to understand reasoning, one must

understand the details of the visual system, motor system, and the general

mechanism of the neural binding.

As evolved creatures, human beings have inherited their ancestors’ capacities

and systems for meeting their needs and coping with a given environment (Brooks,

1990). This tendency to emphasise the continuity between humans and animals, and

the willingness to see instances of intelligent behaviour in animals on evolutionary

grounds has been the motivation behind the study of Embodied Cognition (Gibbs,

2006).

4.1.3 Cognitive offloading

Actions that are used to offload cognitive processing onto the external world in order

to reduce the difficulty of the mental task at hand are called ‘epistemic actions’

(Kirsh & Maglio, 1994). “Examples of epistemic actions include looking at a

chessboard from different angles, organising the spatial layout of a hand of


cards…laying out our mechanical parts in the order required for correct assembly,

and so on.” (Clark, 2008, p. 511)

Problem-solving routines involve intensive computations and internal

representations on one side, and repeated environmental interactions on the other

(Clark, 2008). Clark explains this with an example of using a stick to retrieve a ball

stuck on a roof. People do not measure the distance of the ball or the trajectory of

approach necessary to retrieve the ball. Rather, they try a stick; if it does not work,

they either try another stick, or adjust the length of the first one. People might change

the angle at which the ball is approached, or the direction of approach. They might

re-position themselves to solve the complexity of the visualisation problem, and use

trial and error to decide the appropriate length of the stick.

In a puzzle-solving problem, the manipulation of the puzzle elements through

trial and error serve as tokens to reach a goal state. The manipulation of elements in

the puzzle does not offer any information about a solution to the problem, but it helps

in reaching the goal by revealing clues that could help in making decisions on the

actions to be taken. On the other hand, diagramming represents a different use of the

environment. Here, the cognitive system exploits the external resources to obtain a

solution or knowledge that could actually be used at a later stage. Nathan (2008)

explained the role of gestures in symbolic off-loading. Gestures help to convey

complex and abstract ideas either as a substitute to verbal interactions or as a support

structure to verbal interactions to communicate thoughts and ideas effectively.

Gestures offset the large cognitive demands in verbal interactions that require

communication of ideas and thoughts with clarity while overcoming barriers such as

linguistic, logistic, physical and so on (Hostetter & Alibali, 2008). This type of

symbolic off-loading could be applied to spatial tasks, such as arranging tokens to

represent armies on a map, and could also be applied to non-spatial tasks such as

mind maps to determine logical relations among categories.

Humans exploit the environment to reduce the cognitive load experienced

during ‘on the fly’ cognitive processing because of the limits on attention and

working memory. They use the environment to hold and manipulate information, and

to use it when required. ‘On the fly’ cognition is concerned with immediate input

from the local environment (Iverson & Thelen, 1999, p37). People switch to slower

cognitive processing, relying on representations to make more careful considerations,


such as making a mental check on something odd, or planning future behaviour

(Corr, 2008). Everyday activity such as reading, solving puzzles, and conversations

predominantly use on the fly cognition. However, when the usual flow is interrupted,

people unconsciously switch to slower processing.

Humans are able to handle the representational bottleneck when confronted

with real time cognitive demands. They either use internal representations that they

have acquired through prior learning and experience, or use the environment to

reduce cognitive load. They adapt to changing situations by using epistemic actions

to alter the environment, and reduce the cognitive load in the process (Kirsh, 2013a).

Cognitive processes could be distributed between internal (mind) and external

structures (material or environmental), however this requires a well organised

coordination between these two structures for effective distribution (Cole & Griffin,

1980). Norman (1993) suggested that artefacts improve human abilities in carrying

out tasks, such as the use of calculators to perform arithmetic calculations. Artefacts

can be internal such as task-specific rules used by an expert to perform a task, or

external material artefacts such as GPS used for navigation. Material artefacts are

used when an individual is unable to use internal artefacts. When a person is unable

to perform arithmetic calculations in their mind, he/she uses a calculator to perform

them.

4.1.4 Social Aspect

Knowledge lies not only within the individual, but also in the individual's social

environment (Hutchins, 2000). Activities in the real world are performed through

interactions between people and their tools and artefacts in the context of the task at

hand.

Cognitive processes can be distributed across the members of a social group

(Salomon, 1997). Intelligence can be explained by assembling groups of experts in

various configurations for the execution of tasks. Minsky (1988) suggested that

children learn new concepts through their interaction with others (adults and peers)

and artefacts. The experience of interaction with others enables children to create

functional systems in their absence and, in turn, to contribute towards the creation of

new functional systems in some other person. Minsky (1988) described this as a

transfer of functional skill from one society of mind to another, resulting in a

propagation of a certain pattern in a community.


Dourish (2004) discussed the participatory nature of Embodiment, and

suggested that the social world, along with the physical world, provides context and

meaning to an individual’s activity in the world. Embodied actions are an integral

part of any cooperative interaction (Robertson, 1997). Robertson proposed a

taxonomy of Embodied actions that emerged from observing people doing group

design work. The taxonomy identifies two divisions of Embodied actions, one in

which individual Embodied actions relate to actions on physical objects and spaces,

and another in which group activities are constituted by individual Embodied actions.

Thus, it can be seen from the above description of the cognitive aspects of

Embodiment that cognitive processes are result of elements in the environment,

dynamic processes which are part of the perceptual sensory system of children as

well as the environment and the processes involved in establishing connections

between the mind and the environment. The question then arises how design can

exploit these elements and processes to facilitate intuitive interaction. Section 4.2

explores design aspects derived from the cognitive aspects of Embodiment, which

were further examined in Experiment 1 and Experiment 2 for intuitive interaction,

discussed in Chapter 6 and Chapter 7 respectively.

4.2 DESIGN ASPECTS OF EMBODIMENT

Design aspects of Embodiment were derived from the cognitive aspects of

Embodiment (Section 4.1), which are perspectives of Embodiment from cognitive

science (Anderson, 2003; Thelen, 2008; Wilson & Golonka, 2013; Wilson, 2002)

and from perspectives of Embodiment in interaction design (Shaer & Hornecker,

2010). These aspects are not mutually exclusive, and are non-exhaustive.

4.2.1 Physical affordances

The concept of affordances was originally introduced by Gibson (1979/2014),

who described affordances as the properties of the environment relative to an

individual. The properties are interpreted by individuals through their sensory-

perceptual systems, enabling them to control their actions. For example, the

affordances of climbing a stair in a bi-pedal fashion has been described in terms of

the height of a stair riser in relation to a person’s leg length (Warren, 1984).

Individuals perceive what possibilities for action an environment offers in terms of

its properties, mediums, and compositions. Thus “…the affordances of the


environment are what it offers the animal, what it provides or furnishes, either for

good or ill …”(Gibson, 1986, p.127).

Gibson also emphasised that affordances depend on an individual’s ability to

perform actions in the environment. A chair, for example, affords sitting for adults;

however, for infants it is a walker or a support. Gibson’s affordances does not

depend on the ability of the user to perceive it, and it does not change if the needs

and goals of the user change. McGrenere & Ho (2000); Norman (1999) referred to

Gibson’s affordances as ‘real affordances’ or ‘physical affordances’.

People look for clues in an interface to determine the affordances that it offers

(Dotov et al., 2012). Physical affordances are associated with natural clues, for

example, the weight of a bag. Natural clues represent natural properties of artefacts

such as geometry, shape, or weight of an object. Objects also have properties or

qualities—such as colour, texture, composition, size, shape, mass, elasticity, rigidity,

and mobility—that form their affordances. These physical and material properties

constitute natural clues, and determine and constrain what can be done with the

objects. Physical affordances can be perceived through interactions with objects and

environments in the physical world. Physical affordances in the form of natural clues

embedded in everyday objects and tools, allow real world interactions analogous to

real world services and behaviours (Want, Fishkin, Gujar, & Harrison, 1999).

Gibson introduced affordances in reference to visual perception (Gibson,

1979/2014). However, in Gibson (1966), he explained the role of human perceptual

systems in affordances. He explained the role of senses in perception, especially

focussing on the fusion of information from all senses in perceiving the properties of

objects that would afford action. In recent years, research has focussed on designing

products that allow people to not only use their visual senses, but also their other

auditory, tactile, and olfactory senses (Franinović & Serafin, 2013). This indicates

that natural clues are not necessarily visual only, but can also be sonic, tactile, and

olfactory.

The role of affordances has been discussed in interaction design, specifically in

variations of TEIs (Shaer & Hornecker, 2010). Affordances of interface objects in

TEIs guide the user in how to interact with the computational systems. TEIs rely on

physical affordances to facilitate interactions with computational systems, exploiting

users’ tactile experiences where possible and deriving familiarity from other domains


in the form of metaphorical gestures. Shaer & Hornecker (2010) describe epistemic

actions of Kirsh & Maglio (1994) as an act of looking for clues to interpret

affordances. Constraints are created, new affordances revealed triggerin new actions

and old affordances hidden, this could reduce the complexity of activities.

4.2.2 Perceived affordances

Perceived affordances are based on prior experience with similar things

(Blackler et al., 2010) and, contrary to physical affordances, are learned conventions

(Linderoth, 2013; Norman, 2013). For example, people have been using a

QWERTY keyboard (that maps non-spatial characters in physical space) for decades.

The quality of interfaces that use these kinds of mappings depends on people’s level

of familiarity with the mapping, and on how long they have been using it (Zigelbaum

et al., 2008). (Gaver, 1991) clarified the differences between physical and perceived

affordances, and stated that the latter are ‘often more about conventions than about

reality’.

People use clues, natural and deliberate, to connect them to their past

experiences and prior knowledge. Natural clues such as spatial orientation and

material properties link to previous knowledge of artefacts and their use. For

example stacking blocks look like tall buildings which in turn results in interactions

with the stack that are reminiscent with properties of tall buildings. In the absence

of natural clues (physical affordances), people resort to deliberate clues that are

associated with perceived affordances. Deliberate clues are those that are deliberately

inserted by the designer for specific actions on the artefact. When these actions do

not comply with users’ past experience and knowledge of an artefact, their

interactions with that artefact are non-intuitive. These deliberate clues are symbolic,

and children use their internal representations built from their past experience and

knowledge to decode them.

A popular example discussed by Gaver (1991) is that of a scrollbar on a

computer screen. It not only tells users that they can navigate through the web page

by sliding the scrollbar, but also tells them how much they have read and how much

is still left to read. The scrollbar acts like a virtual book mark to the web page. Scroll

bars have evolved over a number of years, and continue to do so. People learn about

new changes to the scrollbars by acting on them (clicking on them); this, in turn,

results in another clue, such as a double-headed arrow in a word processor. Gaver


(1991) refers to such affordances as ‘sequential in time affordances’ where one clue,

when perceived and acted upon, results in another. Similarly, affordances can be

nested in space; that is, more than one clue for perception action is available at

different places.

Embodied representations are another kind of deliberate clues recommended

by Pezzulo (2011). Embodied representations are not symbolic or linguistic; rather,

they are re-enactments of motor processes in the mind (Pezzulo, 2011). These mental

simulations trigger the same neural processes of a goal-directed action and

perception, however, without any movement and external stimuli. The mental

simulations generate a motor understanding of objects and events in the environment,

and are produced by internal modelling and re-enactment of motor processes. These

mental simulations are triggered by watching something that is in action, such as

bouncing balls. Examples of Embodied representations in design are animated icons

for email in an interface for children developed by Uden & Dix (2000), and

simulations of functionalities of objects. Uden et al. (2000) studied visual and iconic

interfaces for children to facilitate children’s Internet searches. They investigated

animated icons for email, and compared them with icons that were symbolic and

metaphoric. They discovered that children found that it was easy to understand and

recognise the animated icons, compared to the symbolic and metaphoric ones.

4.2.3 Emergence

Emergence in interaction is inspired by the process of evolution of biological

structures. In this context, emergence is the use of existing tools and features in a

machine or an organism to adapt to changes in the environment. These changes could

be either over a short period of time, or over a longer period of evolution. The field

of Robotics saw the need to develop intelligent adaptive systems that were based on

aspects of biological systems (Pfeifer, Iida, & Bongard, 2005), rather than systems

with pre-programmed behaviour. Such adaptive systems are referred to as

‘Evolutionary robotics or ‘Emergent systems’. The final structure of the system is the

result of its history of interaction with the environment.

Emergence enables system adaptation and learning, and provides an indicator

of overall progress in complex systems (Allen & Strathern, 2003). Maier & Fadel

(2009) see emergence as a dynamic property of a system that changes its structure as

people interact with it; and this, in turn, changes the interactions with the system.


Allen & Strathern (2003), however, relate emergence to knowledge that evolves as

people interact with the system; and this, in turn, changes their interactions. Thelen

& Smith (2006), while explaining learning and development in children, suggested

that acquiring knowledge and change in behaviour are inter-related. Children develop

interpretive frameworks based on what they sense and perceive from the

environment. These interpretive frameworks decide their behavioural responses.

Over time, as they develop more knowledge and understanding, they develop more

effective ways to sense and perceive and, in turn, change their interpretive

frameworks. In the meantime, if the environment (physical and social) also changes,

the interpretive frameworks also need to be updated.

Emergence as a design aspect is used in many applications, such as learning

and development, as discussed above (Thelen & Smith, 2006); organisational

management (Senge, 1987); and design sustainability (Jonker & Harmsen, 2012).

The systems in all these applications are treated as dynamic systems with emergent

properties, with a feedback loop from the output of the system feeding back to the

input of the system (Forrester, 1995). The behaviour of a system cannot be

determined from an inspection of its parts as they are connected, and keep changing

their state. This change in state could be due to feedback loops, non-linear

relationships in the system, or behaviour paths, depending on the past history of

interactions. Feedback loops are the main reasons for a system’s emergent properties.

Depending on the relationship between the outputs and inputs in the loop, systems

could develop various behavioural complexities. Jonker & Harmsen (2012), Senge

(1987), and Thelen & Smith (2006) studied emergence in systems by investigating

the feedback loops to determine the behaviour of individual parts of the system; as a

result, they developed design solutions in teaching and learning, sustainability, and

management.

In summary, emergence as an Embodied feature represents the dynamic nature

of the environment, user behaviour, and artefact properties. This dynamism creates a

complex web of design interactions and negotiations, and could be taken into account

when designing interesting interactions and experiences

4.2.4 Scaffolding

Scaffolding is the use of the environment, physical objects, tools, processes,

and support mechanisms to carry out cognitive tasks by offloading some of the tasks


into epistemic actions (Kirsh & Maglio, 1994). Epistemic actions are actions taken in

an environment with the intent of gathering information or facilitating cognition.

Epistemic actions help people to learn things that will inform their future actions, and

to perform more complex tasks (Davis, 2015).

Scaffolding frees the mind from other tasks, resulting in effortless completion

of the primary task. The use of pen and paper to solve numerical problems, and the

use of space and physical objects in solving a puzzle are some of the examples of

scaffolding. Use of physical objects (Clement, 1994) and software tools (Reiser,

2004) are common scaffolds used in education. Physical materials can be extremely

useful in enabling children to construct an essential intuitional foundation for

mathematical thinking and problem solving. For example, children are taught

abstract mathematical concepts through the use of physical objects such as dices and

cubes. For this level of intuition to develop, the physical materials must be used as

the foundation upon which children construct symbolic meanings and operational

definitions. Epistemic actions on these materials and objects allow children to

offload cognitive tasks into activities.

There is scope for investigating ways to incorporate efficient scaffolds in

products and interfaces. A help menu in a software interface, and an instruction

manual of a product, are classic examples of scaffolds that have been in use in design

for many years. Loorbach, Karreman, & Steehouder (2013) proposed the

incorporation of motivational elements in an instructional manual of a cellular phone

for senior users (aged 60–70 years). According to their findings, the use of

verification steps and personal stories in an instructional manual improve: (1)

seniors’ confidence in being able to use the phone; (2) their motivation to work with

the phone; (3) their effectiveness and efficiency in performing tasks; and (4) their

satisfaction with the phone and the manual.

4.2.5 Cooperative activity

Advances in computing and technology have resulted in recognition of the

need for co-operative activity in achieving prescribed goals. Traditionally, physical

objects have been successfully used in facilitating co-operative work (Terrenghi et

al., 2007). Physical artefacts facilitate the well-articulated division of labour (Xiao,

2005). Different people involved in different activities, with the sole purpose of

achieving the same goal, are able to integrate their contribution, access the status


information, and delegate work to the rest of the team, all at the same time. This

happens without intruding upon each other’s activities. Air traffic controllers prefer

paper flight strips over autonomous digital systems to annotate flight information and

share it with fellow air traffic controllers. Interaction with physical objects is visible

to other team members, while interaction with virtual elements in digital systems

may not be visible to all team members. This is very important in high risk, high

pressure situations such as flying an aircraft. Verbal communications form an

essential part of co-operative activity. Africano et al. (2004) found that children

engaged in verbal discussions while playing with children’s artefacts such as

puppets, trading cards, and globes for 40% of the entire activity time, and that this

facilitated co-operative activity.

An underlying objective of TEIs in interaction design has been to facilitate co-

operative activity between people with same expertise and also between people with

varied expertise and skills (multidisciplinary teams) (Shaer & Hornecker, 2010).

Physical artefacts play an important role in starting a conversation among people in a

group. Previous experience and familiarity with the artefacts from everyday life

place less demands on the users to engage with the system. Physical artefacts have

multiple interaction points in terms of action (input to the system) and multiple

observable points in terms of reaction (output from the system). This allows people

in a group to simultaneously interact with the system as they all can act on the

artefact at the same time and the outputs from the system are observed by everyone

in the group. This results in group awareness and coordination (Shaer & Hornecker,

2010). However, careful design of TEIs is required for appropriate embodied

facilitation. Jordà, Geiger, Alonso, & Kaltenbrunner (2007) designed ReacTable with

a circular shape to encourage co-operative activity. Circular shape encouraged people

to move around the table and turn the table like a Lazy Susan (Vanity Fair, 1917) is

used distribute food to people.

4.3 RELATIONSHIP BETWEEN COGNITIVE AND DESIGN ASPECTS OF EMBODIMENT

The relationship between the cognitive and design aspects of Embodiment is

shown in Figure 3. The lightly shaded (in black font) rectangular boxes represent the

cognitive aspects of Embodiment, and the dark shaded (in white font) rectangular

boxes represent the design aspects. The double arrowed lines connecting two


rectangular boxes represent the relationship between the cognitive and design

aspects.

Figure 3 Design aspects of Embodiment derived from cognitive aspects

Physical affordances and perceived affordances are related to the real time

cognitive aspect. This is because the primary objective of affordances is to offer

appropriate clues to perform actions that are performed in real time. It is not normal

to interpret clues and perform actions in the future. Physical affordances are also

related to the real world because they are associated with use of natural clues to

interpret the actions to be performed on the world. These natural clues exist only in

the real world, with real products with natural properties. Perceived affordances, on

the other hand, are not necessarily always used in the real world. Perceived

affordances are often used in virtual worlds through deliberate clues to assist in the

interpretation of actions to be performed. The dotted line in Figure 3 represents these

dual possibilities of perceived affordances in terms of linkages with the real-world

aspect. This is also in line with Dennett's (1991) view that narratives of the past and

past experiences and memories could influence the intentions of an individual.

Physical affordances are related to cognitive offloading, as actions performed on


natural physical objects free the mind from cognitive processes and tasks. Activities

performed with physical and material elements assist the information that was solely

processed in the mind.

Scaffolding, which in literal sense means offering support to a structure, is

related to cognitive offloading. Emergence is linked to the evolutionary aspect of

design as it is associated with changes to the properties of artefacts and environments

and, in turn, to the behaviours and interactions of children over a period of time. In

other words, the properties of the artefacts, environments, behaviours, and

interactions of children in time are due to the dynamic features of the artefacts and

environments which they act upon.

The relationship between cooperative activity and the social cognitive aspect is

obvious. Products are increasingly required to support children’s cooperation with

others (e.g. having children and adults involved in an activity). Research in the field

of Computer Mediated Communications (CMC) and Computer Supported Co-

operative Work (CSCW) has been carried out over three decades. However, most of

this work focussed on remote collaborations, where people in a group are dispersed

geographically or spatially (e.g. the use of Massive Open Online Courses (MOOCs)

in teaching and learning (Howarth, D’Alessandro, Johnson, & White, 2016).

However, the success of this approach is debated in the literature (Padilla Rodriguez,

Armellini, & Caceres Villalba, 2016). Children work and play in groups, and

children’s products need to support this co-operative work. It is evident from the

above discussion that the co-operative aspect of Embodiment is in line with its social

aspect.

Thus, Embodiment represents aspects of human interaction in the real world. It

includes aspects of human evolution, relating our changes in abilities and our

understanding of the world to our epistemic actions. The design aspects of

Embodiment were further investigated and examined for intuitive interaction in

Experiment 1 (Chapter 6) and Experiment 2 (Chapter 7).

4.4 SUMMARY

This chapter discussed cognitive aspects of Embodiment. These aspects are

based on the perspective of Embodiment that all cognitive processes are result of a

continuous loop of perception and action that involves brain, body and environment


playing an important role in a complex mesh. These perception action loops originate

from the environment and not the brain (Anderson, 2003; Wilson & Golonka, 2013;

Wilson, 2002). It is the stimulus from the environment that triggers perceptual

sensory systems in children. Design aspects of Embodiment (Section 4.2) were

formed from the literature on Embodied cognition. The cognitive aspects of

Embodiment were discussed (Section 4.1) and the relationship between the design

and cognitive aspects of Embodiment discussed in Section 4.3.

The design aspects of Embodiment were further examined and verified in

Experiment 1 (Chapter 6) and Experiment 2 (Chapter 7). Chapter 5 discusses the

research design for this study, its data collection and analysis methods, and the

rationale for their choice.


Chapter 5: Research Design

This chapter outlines the research design and methodology adopted by this

study to achieve the aims and objectives stated in Chapter 1. Methods used for data

collection and analysis are explained in relation to the research questions, with a

focus on the rationale for the choice of these methods.

Section 5.1 discusses the study’s methodology and research design, briefly

describing the two experiments conducted; Section 5.2 details the participants in the

study; Section 5.3 discusses the data collection methods, and justifies their choice;

Section 5.4 outlines the various data analysis tools and methods and explains how the

results were integrated to report the findings and conclusions.

5.1 METHODOLOGY

The objective of this research was to investigate the role of Embodiment in

children’s intuitive interaction. Physical products, virtual interfaces, and TEIs were

studied for children’s Embodied intuitive interaction with them. The research

questions that formed the basis for the research design were:

What is the role of Embodiment in children’s intuitive interaction?

What are the aspects of Embodiment?

To what extent do the design aspects of Embodiment facilitate intuitive interaction

in children?

How can the design aspects of Embodiment facilitate children’s intuitive

interaction?

A mixed methods approach to research design was used. Mixed methods

research uses a combination of qualitative and quantitative approaches to data

collection, analysis, and interpretation of findings. Empirical research that uses

mixed methods either focusses on collecting and analysing two types of data

(qualitative and quantitative), or on integrating its qualitative and quantitative

findings (Tashakkori & Creswell, 2007; Zhang & Creswell, 2013). Thus, in an effort

to be inclusive of both qualitative and quantitative approaches, Tashakkori &

Creswell (2007) defined mixed methods research as “research in which the


investigator collects and analyses data, integrates the findings and draws inferences

using both qualitative and quantitative approaches” (p. 3).

Mixed methods research draws upon the strengths of each of the qualitative

and quantitative approaches and was particularly useful for this thesis as it offered

various perspectives on the study of the complex phenomena of Embodiment and

intuitive interaction. Integration of qualitative and quantitative findings at some stage

of the research process (i.e. at the data collection, analysis, or interpretative stage) is

a critical characteristic of mixed methods (Kroll & Neri, 2009).

In this study, qualitative data were collected through observations, co-

discovery methods, and retrospective interviews. Thematic analysis was performed

on the qualitative data, which was then used to generate quantitative measures. The

quantitative measures were then statistically analysed. The findings from the

thematic and statistical analyses were integrated in the interpretative stage of the

research process.

The research design consisted of two sequential experiments, the outcomes of

Experiment 1 informing the design of Experiment 2. Both the experiments involved

children as participants. Children have been involved in previous research studies at

different levels. Wang (2015) identified that traditional studies involving children,

especially in psychology, were mostly of the non-contact type. Data was collected

from administrative records and from people (such as parents, teachers, and carers)

who were in contact with children. In recent years, however, children have started

playing more of a participatory role in research. Druin (2002) and others have

considered children’s direct participation in the design process of technology

specifically designed for them. Liamputtong (2006) found that direct contact research

mostly involves self-completion questionnaires, interviews, and group discussions.

Furthermore, observations have been mostly limited to non-interactive data

gathering. In contrast, children were participants in this study, playing with toys with

another child (and, in the first study, with the researcher).

Ethics approval using the National Ethics Application Form (NEAF) was

sought from the Human Research Ethics Committee (HREC) at QUT prior to any

data collection (including the pilot studies). An approval from Education

Queensland was also obtained to conduct study at a local state school. The ethics

approval from HREC, participant information sheets, consent forms and approval


from Education Queensland is placed in Appendix (Appendices F-O). The structure

of the entire study, with details regarding each experiment, is shown in Figure 4.

Both the experiments were conducted in a similar way, with minor changes

made to Experiment 2, based on the findings from Experiment 1. Both experiments

involved pairs of children playing with real toys. Trial experiments with children of

friends and family were carried out prior to Experiment 1 to decide on: how children

should be paired; the selection of appropriate toys for the experiment; the

combination of data collection methods; and protocols for communicating with the

children. To cover the social aspect of Embodiment in the investigation, children

were observed playing a game with another child. The fact that children were playing

with another child whom they knew before the experiment also assisted in making

them feel comfortable during the study. This was important for the study to bring out

natural interactions and behaviours during the play. Experiments were audio and

video recorded for analysis.

5.1.1 Experiment 1

Experiment 1 was a between subjects study that investigated the aspects of

Embodiment that facilitate intuitive interactions with physical products and virtual

interfaces. The objective of the experiment was to determine the role of Embodiment

in children’s intuitive interaction with a directly manipulated interface and a virtual

interface. Their intuitive interaction with, and aspects of Embodiment that facilitated

their intuitive interaction with a physical product and an equivalent virtual interface,

were compared. A summary of Experiment 1 is presented in Table 2.

Observations that involved both children and the researcher in game play were

carried out in Experiment 1. Children who knew each other outside the experimental

setting and were around the same age (or from the same class or grade at school)

were paired for the experiment. Both the children played as a team against the

researcher. Observations were followed up with retrospective interviews, without any

concurrent protocols. Both observations and retrospective interviews were audio and

video recorded for analysis. Experiment 1 was carried out at a local state school in

Brisbane (Australia), and in the People and Systems Lab (PASP) at Queensland

University of Technology (QUT).


Figure 4 Research Design to investigate children’s Embodied intuitive interaction


Table 2 Summary of Experiment 1

Experiment 1 compared intuitive interaction and Embodiment between

physical product and virtual interface. A between subjects study was more

appropriate for this type of study as it allowed the experiment to be conducted with

two independent groups of participants without the carryover effects; that is,

participation in playing with one toy does not affect performance in playing with the

second toy. Practice or fatigue while playing with one toy, for example, could either

enhance or stifle performance in playing with the second toy. This results in a

confounding variable that can vary with the independent variable (Gray and Kinnear,

2012, p. 316).

5.1.2 Experiment 2

Experiment 2 was a within subject study that investigated ways in which

Embodiment can facilitate children’s intuitive interaction when playing with directly

manipulated interfaces. A summary of Experiment 2 is presented in Table 3.

With the exception of a few changes, the experiment was carried out in the

same way as Experiment 1. A Within subjects study design was used in Experiment

2, and each pair of children played with two directly manipulated interfaces—one

was a TEI, and the other a physical product. The TEI consisted of a physical space

and a virtual space, and involved direct manipulation and interaction with the

physical elements in the physical space. Data collected for each of the toys were

analysed separately (not compared with each other as in Experiment 1). A between

subjects study requires a large sample size. Recruitment of a large number of

participants, data collection and data analysis for this large sample is time

Experiment 1 Research Question Can Embodiment facilitate intuitive

interaction in children?To what extent do the design aspects of Embodiment facilitate intuitive interaction in children?

Setting QUT People and Systems Lab and Queensland State Schools

Experiment Design Between subjects Data Collection Methods Observations, Retrospective Interviews,

Co-discovery Data Analysis Thematic and statistical analysis Population School children Sample/Participants 108 (54 pairs) children aged 5–11 years


consuming. Within subject designs, on the other hand, require smaller sample size

for approximately same effect size and statistical power (Section 5.2.1 and Section

5.2.2). A smaller sample size means participant recruitment, data collection and

analysis comparatively takes less time. With significant amount of time spent on

Experiment 1, a within subjects design was used for Experiment 2.

Table 3 Summary of Experiment 2

Within-subject designs suffer from carry-over effects, Gray and Kinnear (2012)

suggested using counter-balancing to reduce the possibility of carry-over effects

confounding the effects of the game play. Thus, the order in which a child pair

played each of the games was varied from one observation to another, ensuring that

there were equal numbers of pairs playing each game first. This way, any carry-over

effects balanced out across the two game plays.

Since every child played with two toys, retrospective interviews were not

carried out after Experiment 2 in order to keep the experiment time reasonable for

the participants. Pilot studies performed prior to Experiment 2 revealed that children,

especially younger children, got tired and lost focus while playing for a lengthy

period. Furthermore, communication between the children during the observations

generated enough verbal data for analysis.

Again, children were paired if they knew each other outside the experimental

setting. However, unlike Experiment 1, they were not necessarily of the same age.

Children normally play with other children who may not be of same age such as

siblings, play mates outside the school environment such as at a sports club. Children

of same age were paired for Experiment 1 as it was a comparative study and the

Experiment 2

Research Question - Which design aspects of Embodiment facilitate children’s intuitive interaction - To what extent do the design aspects of Embodiment facilitate intuitive interaction in children?

Setting QUT People and Systems Lab

Experimental Design Within subject

Data Collection Methods Observations and co-discovery

Data Analysis Thematic and statistical analysis

Population School children

Sample/Participants 42 (21 pairs) children aged 5–12 years


difference in ages could have skewed the results. Intergenerational research

involving participants from different age groups has been successfully used in the

Social Sciences (Grenier, 2007), and in the design of technology products for

children (Guha, Druin, & Fails, 2013). However, that research often focussed on the

role of adults (over 50 years of age) as mentors to the children. In contrast, in order

to mimic natural play in children in Experiment 2, children were paired so that they

were not more than 4 years apart in age, and were paired with a sibling or a friend.

Experiment 2 was carried out at the PAS Lab, and participants were recruited

through advertisements in local state schools.

5.2 PARTICIPANTS

Children in the 5–12 age group were selected from a pool of volunteers from

local state schools in Brisbane (Australia), and through personal contacts. These

volunteers were recruited through school newsletters and email advertisements sent

out to email groups at QUT. Parents/guardians/carers and children filled out consent

forms prior to the experiment. Every participant was provided with a gift card or a

toy as a gesture of thanks for their participation. Children were asked to choose a toy

from a reward basket at the end of Experiment 1, and gift cards were given to the

parents in Experiment 2.

Children usually start school in Queensland, Australia at the age of 5-6 years

(Prep) and make a transition to middle school at the age of 12 years (Grade 6). This

transition brings changes in children’s behaviour (Heyns, 1987) and how they

approach a given problem and the way they process information cognitively (Rutter,

1985). Perry, Church, & Goldin-Meadow (1988) discussed how these transitional

stages bring a change in the acquisition of concepts. This explains the choice of the

age range 5 years to 12 years for the study and it was important to include the years

of the transitional stages in the study. Prep and Grade 6 represent a transitional

change in knowledge and concept acquisition in children in Queensland schools.

This explains the large age range (5 years – 12 years) in this study.

The number of children recruited for the study depended on the design of the

experiment (between-subjects in Experiment 1, and within-subjects in Experiment 2),

and statistical tests employed for quantitative analysis (parametric or non-parametric,

independent groups or matched pairs, etc.). A high statistical power meant that the


probability of making a Type II error, or concluding that there is no effect when there

is an effect present, is less. For Experiment 1 and Experiment 2 a power analysis was

used to determine the prospective statistical power of the experiment design and thus

the implied probability of making a Type II error. The probability of a statistically

significant result is high if the effect size (f2), sample size (N) and the significance

criterion (α) are high.

Optimal statistical power chosen for the study was 0.8. A statistical power of

0.8 means that the probability of rejecting a false null hypothesis is 0.8 or 80% or we

run a 20% chance of making a Type II error. Significance criterion, α = 0.05 has

been used in this study for all the analysis, including power analysis. G*Power, a

software for statistical power analysis (Faul, Erdfelder, Buchner, & Lang, 2009) was

used to estimate effect size and statistical power for Experiment 1 and Experiment 2.

5.2.1 Power analysis for Experiment 1

G*Power was configured for non-parametric between subject test for two

independent groups. Power analysis was run with the following parameters:

Initial Sample size (N1) for group1 – children playing with physical toy: 50

Initial Sample size (N1) for group2 – children playing with virtual toy: 50

α = 0.05

Estimated lowest effect size using G*Power from the mean values of the

dependent variables in Experiment 1 (discussed in Section 6.2.2), f2 = 0.7

Corresponding Power = 0.937

XY plot for computed power for sample size varying from N = 0 to 100 was

plotted to assist in the final selection of the sample size (N) (Figure 5). The statistical

power for an initial sample size of 50 for each of the independent groups, N1 = N2 =

50, was greater than the optimal power of 0.8, so the sample size was considered

appropriate for Experiment 1. The total sample size selected for Experiment 1 was

108, N1 = 56 and N2 = 52, contributing to a statistical power of 0.945.

5.2.2 Power analysis for Experiment 2

G*Power was configured for non-parametric within subject test for matched

pairs (pairs of children playing with both the toys). Power analysis was run with the

following parameters:


Initial Sample size (N): 50

α = 0.05

Estimated lowest effect size using G*Power from the mean values of the

dependent variables in Experiment 2 (discussed in Section 7.2.2), f2 = 0.52

Corresponding Power = 0.945

XY plot for computed power for sample size varying from N= 0 to N= 100 was

plotted to assist in the final selection of the sample size (N) (Figure 6).

Figure 5 XY plot of power for a range of sample size (N) value, 1 to 100

Figure 6 XY plot of power for a range of sample size (N) value, 1 to 100

The statistical power for an initial sample size of N=50 was greater than the

optimal power of 0.8, the sample size was considered appropriate for Experiment 2.


The total sample size selected for Experiment 2 was 42, contributing to a statistical

power of 0.925

5.3 DATA COLLECTION METHODS

Data were collected using observations, co-discovery, and retrospective

interviews.

5.3.1 Observations

The objective of the study was to study intuitive interactions in children,

interactions that are Embodied in their everyday life and activities. Physical artefacts

are mediators between the agent and their activity of manipulating these artefacts

(Nardi, 1996). This manipulation of artefacts brings out natural behaviour in the

agent (Engeström, Miettinen, & Punamäki, 1999). Thus, artefacts can be used to

study people’s complex behaviours (Popovic, 2003), as they result in human

activities that are representative of user goals. Thus, both experiments in this

research study involved performing activities with an everyday artefact.

Play as an activity comes naturally to children, and is part of their daily

activities. Thus, playing with toys with other children formed the basis of the

experiments to investigate children’s Embodied intuitive interaction. Children learn

how to use different features in a product by exploring and tinkering with it

(Blikstein, 2013). Adults, on the other hand, create a step by step strategy in their

minds, even before they begin a task or touch a product (Fisk, Rogers, Charness,

Czaja, & Sharit, 2009). Thus, children were given the holistic task of playing a game;

this allowed them to explore and invent new strategies of game play.

Children’s play was audio and video recorded for thematic and quantitative

analysis. Two video cameras were used to record the observations. One camera

captured the front view of the children playing with the toy. The second camera

either captured the screen view of the app on the tablet, or the side view of the play

with the physical toy (depending on whether there was a tablet involved in the play).

5.3.2 Co-discovery

Observations alone might not provide insight into the decision making process

involved in children’s intuitive thinking. Thus, observations were complemented

with verbal protocols to capture information that could not be seen externally.


Concurrent protocols (also known as ‘Think aloud’ and ‘Talk aloud’ verbal

protocols) have been successfully used in empirical studies of intuitive interaction as

they offer an immediate account of thoughts and actions either without, or with

minimal prompts (Blackler et al., 2010). Concurrent protocols, however, can be

difficult for children when they are interacting with systems that require a heavy

cognitive load (Höysniemi, Hämäläinen, & Turkki, 2003). Children’s cognitive

abilities are not fully developed until their late teens, and this influences their ability

to think and talk out loud. For example, they have a limited ability to think

simultaneously about more than one concept, such as playing with a toy and thinking

aloud at the same time.

Concurrent protocols make the conditions in an experiment unnatural, and are

difficult to facilitate in a field setting (such as schools) or in a collaborative context

(Hertzum, 2016). Trial experiments with children of friends and family revealed that

although instructed to think and talk aloud, they remained engrossed in the play.

They were not clear about what to do when they were instructed to think or talk

aloud. When prompted to verbalise their feelings and thoughts, they stopped playing

and talked. They also talked about things that were irrelevant to the game. When they

went back to play, their game strategies were different to those they employed before

the interruption. They took longer to finish the game, reacting slowly to various

stimuli (visual and haptic). They also spent a lot more time in distributed visual

behaviour, looking for something to talk about. This confirmed that concurrent

protocols affect children’s performance of spatial tasks (Gilhooly, Fioratou, &

Henretty, 2010). This means that it is difficult to use them in the context of play.

Considering the above problems with concurrent protocols, co-discovery was

used as a verbal protocol. Each child participant was paired with another to solicit in-

depth information and free-flowing discussion between them (Adebesin, De Villiers,

& Ssemugabi, 2009; Kemp & Van Gelderen, 1996). The verbalisation of ongoing

thought processes provided direct insights into the knowledge and methods used

during the play (Popovic, Kraal, Blackler, & Chamorro-Koc, 2012). This verbal

discussion between participants while performing a task is referred to as ‘co-

discovery’ (Lim, Ward, & Benbasat, 1997).

Co-discovery encouraged children to question each other and to engage in

deeper discussions and explanations; that is, to consider the why and how of the task.


It ensured that the researcher’s presence did not influence the children’s behaviour.

Co-discovery provided a platform for the children to discuss and question every

strategy in the game; this reflected not only what they were thinking, but also why

they were thinking it. This natural verbal communication between the children

provided rich verbal data that represented the internal cognitive processes that were

mapped into their interactions with the game system. They brought their own

experiences and knowledge into the discussion. The two sets of knowledge and

experiences provided a critical perspective on the understanding of the children’s

interactions.

The success of co-discovery depends largely on how participants are paired.

The level of expertise between participants is crucial for pairing them in co-

discovery (O’Malley, Draper, & Riley, 1984). Nielsen (1994) favoured pairing

participants with the same level of expertise in co-discovery so that expertise of any

one participant does not influence the outcomes of the study. Kahler, Kensing, &

Muller (2000), on the other hand, suggested pairing participants with different levels

of expertise, thus enabling one person to guide the interaction. However, pairing

children with different levels of experience could result in a child with higher levels

of experience in taking over the game play. Level of acquaintance is another

important element that plays an important role in pairing participants in co-discovery

(Als, Jensen, & Skov, 2005). Children often behave differently depending on how

well they know each other. Rather than expertise, level of acquaintance was more

important for this research study as it made children comfortable and encouraged

natural behaviour. Children who knew each other outside the research study - as

friends, classmates, or siblings - were paired for the experiments in this research.

Experiment 1 was mostly conducted at a local school in Brisbane, and thus

children from the same class or grade were paired together. This ensured that

children were acquainted with each other. This also made sure that children with the

same level of expertise were paired. The school where the data was collected

introduces the physical toy and the app to students in Prep. Pairing children from the

same grade (and who have been introduced to both the toys in Prep) ensured that

both the children had the same level of familiarity. Experiment 2, on the other hand,

was conducted at People and Systems Lab (PAS Lab). Children and their parents

were invited to participate in the study and were asked to bring along a friend or a


sibling whose age was not more or less than the child’s age by 4 years. This firstly

ensured that the children knew each other outside the study. Secondly, the

requirement of age difference not more than 4 years ensured that none of the children

in the pair were overly expert compared to their playmate.

Gorriz & Medina (2000) suggested that gender influences pairing in co-

discovery. Girls typically have different preferences than boys when it comes to

playing with toys. However, this has been debated by many who suggest that a

preference for specific type of toys depends on the influence of family members who

direct children towards specific types of toys and activities (Alexander, 2003).

Children also change their preferences depending on their exposure to specific toys.

Caldera, Huston, & O’Brien (1989) further suggested that toys and games can be

designed to be gender neutral and thus gender was not considered as criteria for

pairing children.

5.3.3 Retrospective Interviews

Retrospective interviews were carried out after the game play in Experiment 1.

Children were shown the video of their game play, and the verbal discussions

between them were recorded for analysis. However, they were neither instructed nor

prompted to think or talk aloud at any time. Co-discovery was used as a verbal

protocol in retrospective interviews, as it was in the observations.

Conscious events immediately fade in memory, and children might not

remember the details of game play during the interview (Popovic et al., 2012). They

were thus shown the audio and video recording of their play on a monitor screen

immediately after their play session. They talked about their play, and about why and

how they made their game decisions. They also asked their experiment partners

questions about the game. Researcher intervention in terms of prompts or questions

was required only when there was absolute silence; however, this was very rare.

Retrospective interviews were only carried out in Experiment 1. They were

video recorded, using two cameras. One video camera recorded the front view of the

children sitting together and discussing their play. The second video recorded the

view of the screen monitor on which they were shown the game play.


5.4 ANALYSIS

The audio and video recordings of observations and retrospective interviews

were analysed in three steps. The data was first coded for Embodiment and intuitive

interaction, using thematic analysis. Coded data was then exported to Excel for

quantitative analysis, using SPSS and STATA.

5.4.1 Thematic analysis

Thematic analysis was used to identify, analyse, and report patterns (referred to

as ‘themes’) within the data. A deductive approach to thematic analysis was used.

The analysis was carried out in Noldus Observer XT (Version 12) (Noldus, 1989),

where two videos from the observations and two videos from the retrospective

interviews (i.e. four videos) were all synchronised and analysed. This linking of the

verbal discussions (captured through co-discovery in observations) and the

retrospective interviews to the actions performed on the toys, allowed a better

understanding of the children’s interactions and behaviour.

Two groups of themes, and their corresponding sub-themes, identified based on

the research question and the literature review, are shown in Figure 7. Thematic

analysis was carried out in four stages to identify the interactions and behaviours that

were to be coded, and the heuristics for the themes. ‘Heuristics’ refers to strategies

derived from experience-based techniques for problem solving, learning, and

discovery (Pearl, 1984). The heuristics for this research study partly came from the

literature review, and partly from the data. This was important in applying the

heuristics to the context of the data. It was necessary to move back and forth

throughout the four stages of thematic analysis.

Data familiarisation: Direct involvement in the data collection process

provided some initial knowledge about the characteristics of the data. Videos of

observations and retrospective interviews were watched multiple times to become

familiar with them, and patterns of interest emerged as a result. Interesting

interactions and behaviours of the children relevant to the research questions, and

quotes from the verbal interactions between them were noted. This familiarisation

and notetaking was followed by a formal process of coding.

Initial coding stage: Initial codes for the children’s behaviour, and their

interactions with the toys and their partners, were generated in Observer XT. Extracts


of data were coded inclusively. The initial codes were in the form of comments,

quotes, and words relevant to the extract of data being coded. The heuristics for

coding interactions and themes began to emerge from the initial codes.

Formative stage: Initial codes that represented common themes were grouped.

Some initial codes formed the interactions; some formed the themes; and some did

not fit into either category. These were categorised as ‘others’. None of the initial

codes were discarded at this stage.

Review stage: In this review stage, the themes identified in the previous stage,

and the relationship between the themes and interactions were reviewed for

coherency, overlapping themes, missing behaviours or interactions, and correctness

of the codes. The heuristics were also finalised in this stage.

Figure 7 Coding scheme showing theme groups and the corresponding sub-themes

Definition stage: This was the final stage, where the coded data were reviewed

again, and the heuristics for the interactions and themes were finally clearly defined

in the context of the games played within each of the experiments.


The coding heuristics for Types of interaction and Aspects of Embodiment are

discussed below.

Coding Heuristics for Types of interaction

There were three themes within the theme group, types of interaction: intuitive,

non-intuitive, and partially intuitive. These themes were both mutually exclusive and

exhaustive.

Intuitive Interaction involves utilising knowledge gained through other

experience(s); it is fast, and generally unconscious (Blackler et al., 2010). The coding

heuristics employed to code for the intuitive interactions are based on this definition,

and are derived from methods developed by Blackler (2008). The main indicators of

intuitive interaction that were used to decide on the types of interaction during the

coding process are explained below. Two of these conditions needed to be met for a

behaviour to be coded as ‘intuitive’.

Unconscious Reasoning- Intuitive interaction is associated with processes that

yield little or no conscious experiences (Bastick, 1982; Blackler, 2008; Dane & Pratt,

2007; Myers, 2007). This means that if children demonstrated less reasoning while

playing with the toy, they could be interacting with it intuitively. Unconscious

reasoning is associated with mental states that are not verbally reportable (Chalmers,

1995). People are unable to access their internal mental states to verbally report

about them. Baars, Ramsøy, & Laureys (2003) studied the activity in the human

brain during conscious and unconscious mental states. They reported that

behaviourally conscious states demonstrated accurate reportability of attended

stimuli, orientation to space, time and self, visual images, inner speech, abstract

thoughts, and control of voluntary muscles. Unconscious states, on the other hand,

did not demonstrate any accurate reportability. In terms of brain activity, low

regional metabolism was observed in the frontoparietal cortex in unconscious mental

states. On the other hand, high regional metabolism was recorded in the

frontoparietal cortex in conscious mental states. The frontoparietal cortex is the

“…narrative interpreter of the speaking hemisphere…” (Baars et al., 2003, p. 673) of

the brain that is responsible for the reportability of the stimuli and events.


Children who were unable to accurately explain their game play—that is why

and how they decided on their game strategies—were considered to be in a

nonconscious mental state, processing information intuitively.

Degree of Certainty- Intuitive interaction is associated with high degrees of

certainty, confidence, and expectation with respect to the correct use of a feature

(Blackler, 2008; Hammond, 1993; Simmons & Nelson, 2006). Play is usually not

associated with correctness or incorrectness. Thus, when participants were certain

and confident about their decision and their interaction with the toy, the behaviour

could be coded as ‘intuitive’.

Certainty and confidence in children’s play was evident from their interaction

with the toys. Children would not hesitate, and their interactions would be

immediate. Certainty and confidence was evident from their facial expressions and

body movements. Ekman, Friesen, & Ellsworth (2013) provided guidelines to study

emotions and behaviours in people based on their facial expressions. They described

findings from experiments performed by Jones (1971), Ekman (2006) and Hulin &

Katz (1935). Of particular relevance are findings of Jones (1971) who investigated

facial expressions in children and tabulated behaviours and meanings for singular,

and combinations of expressions. Jones suggested that expressions such as biting

fingers, frowning, raising upper brows, and closing eyes represented uncertainty and

lack of confidence in carrying out a given task. Mahmoud & Robinson (2011)

studied gestures in children, and suggested that folding hands, rubbing hands

together, twitching fingers, and placing hands against their face are characteristics of

uncertain behaviour. Verbal conversations in co-discovery with a partner could also

provide indications of uncertain behaviour.

Fast decision making- Intuitive decision making is associated with fast

decisions (Plessner, Betsch, & Betsch, 2011). If a participant made a decision within

five seconds, the decision was considered to be fast, irrespective of whether it

resulted in successful play or not. However, a fast decision was coded as ‘intuitive

interaction’ only if the decision was taken with a degree of certainty and unconscious

reasoning.

Non-intuitive interactions are associated with decisions that are made

consciously. This means that if children were reasoning enough while playing with

the toy, they could be interacting with it non-intuitively. Conscious reasoning is


associated with mental states that are verbally reportable (Chalmers, 1995).

Behaviourally conscious states demonstrate accurate reportability of what trigged an

action, elements in the environment such as images, objects and their orientation in

space, mental thoughts, abstract concepts and control of their physical body (Baars et

al., 2003).

Non-intuitive interactions are associated with high degrees of uncertainty and

lack of confidence (Hammond, 1993; Simmons & Nelson, 2006). Thus, when

participants were uncertain and lacked confidence about their decision and their

interaction with the toy, the behaviour could be coded as ‘non-intuitive’. Evidence of

hesitation in their interactions with the toys was used to determine uncertain

behaviour. Ekman, Friesen, & Ellsworth's (2013) guidelines to determine level of

uncertainty and confidence from facial expressions and body movements were used

to determine non-intuitive interactions. Trial and error such as random playing with

physical objects, random tapping and swiping on touch screen interfaces is

associated with uncertainty and lack of confidence (Chin, Diehl, & Norman, 1988).

Thus, trial and error interactions with the toys were coded as non-intuitive

interactions.

Partial intuitive interactions: Children’s game play at times consisted of

interactions that showed signs of both intuitive and non-intuitive interactions.

Children could for example figure out the use of certain colour coded objects in the

game unconsciously and quickly but at the same time demonstrate uncertainty in the

use of some other colour codes. Such behaviour was coded as ‘partial intuitive

interaction’.

Coding Heuristics for aspects of Embodiment

The themes identified within the theme group Aspects of Embodiment were

physical affordances, perceived affordances, emergence, scaffolding, and co-

operative activity. These themes are the design aspects of Embodiment discussed in

Section 4.2. These themes were not mutually exclusive, and were non-exhaustive.

Physical affordances: posit that people directly perceive and act in the

environment through the ability of their perceptual sensory systems to detect

information about the environment. Physical affordance is a way of deriving

information from the environment when the body and environment are involved in a


perception action loop. People use natural clues from the environment such as spatial

and material properties of objects to derive this information from the environment.

These natural clues determine and constrain what can be done with the elements in

the environment. When children used natural clues from the objects in the

environment to play with the toys, the behaviour was coded as ‘physical

affordances’.

Perceived affordances: posit that people use information from their past

experience and prior knowledge to decide on actions to be performed on the

environment. This is in contradiction to physical affordances as the perceptual

sensory systems are completely removed from the process. While designing products

that tap on people’s past experience and prior knowledge, deliberate clues are

incorporated in the design of products. People use these deliberate clues to make

connections between the problem at hand and their knowledge and experience. As

discussed in Section 4.2, there are two types of deliberate clues, symbolic clues and

embodied representations. Symbolic clues are the conventional ones which have

been used in design for decades. They represent symbols in the form of language,

semiotics, aesthetics, cultural representations, etc. Embodied representations are

clues in the form of actions that offer meaning to the object or the design element.

When children used deliberate clues to play with the toys, the behaviour was coded

as ‘perceived affordances’.

Emergence: is a dynamic property of a system that changes its structure and/or

behaviour as people interact with it; and this, in turn, changes the interactions with

the system (Maier & Fadel, 2009). As the system and interactions adapt with each

other, people learn and develop knowledge about the dynamic system (Allen &

Strathern, 2003). Interactions and behaviours which led to changes to the structure of

the game or the toy resulting in changes to game strategies, were coded as

‘Emergence’.

Scaffolding: is the use of environment, physical objects, tools, processes, and

support mechanisms to perform cognitive tasks by offloading some of the tasks into

epistemic actions (Kirsh & Maglio, 1994). When children were using elements from

the environment such as physical objects, processes (such as imitation) and support

mechanisms (help menu for example), the behaviours were coded as ‘scaffolding’.


Co-operative activity: is the idea of working together to reach a particular

goal. This could be through use of physical objects or through verbal communication

between children. It is known fact that actions performed on the environment by a

person alone are different to actions performed by a group (Williams, Kabisch, &

Dourish, 2005). It is this behaviour of working together in an attempt to reach a goal

was coded as Co-operative activity.

5.4.2 Reliability analysis

A reliability analysis of the coded data was required to verify that the coding

was repeatable. The data were coded twice by the same researcher, with a break of

15 days between the two sessions. Reliability analysis was carried out on the two

versions of coded data in SPSS.

The strength of agreement between the two versions of coding was reported

through Intraclass Coefficients (ICC) for reliability which can range from 0 to 1. Koo

& Li's (2016) guidelines (Appendix A) were used to assess the strength of

agreement.

5.4.3 Quantitative analysis

All coded data were exported to Excel for analysis. While SPSS was primarily

used for statistical analysis, STATA was also used to determine sample size (that is,

the number of participants) for both experiments, and to assess the pre-requisites for

the regression analysis methods used in Experiment 2.

This research study used statistical methods to determine if differences

between groups existed, and to predict the variability in intuitive interaction resulting

from the design aspects of Embodiment. Descriptive statistics were first used to

summarise data within individual groups in such a way that patterns could emerge.

The results were reported in the form of tables and graphs (i.e. bar graph and box

plot).

The significance of the differences between the two groups was determined by

the use of inferential statistics. The quantitative data in Experiment 1 and Experiment

2 did not have the requisites for parametric inferential methods; thus, non-parametric

methods for significance testing were used. The Mann Whitney U test was used for

Experiment 1 and the Friedman test for Experiment 2. The non-parametric inferential

statistical test only tells if the differences between the groups are significant. Thus,


effect size (also called ‘Cohen’s d value’) was reported to tell the size of the

difference. Cohen’s (1992) guidelines (Appendix B) were used to determine the size

of the difference.

Multiple regression analysis was used in Experiment 2 to explain how much of

the variation in intuitive interaction could be explained by the aspects of

Embodiment (i.e. predictors): physical affordances, perceived affordances,

emergence, scaffolding, and cooperative activity. The objective was to investigate

the relationships between intuitive interaction and these aspects. Multiple regression

analysis requires observations to be independent, and data to be free of

multicollinearity (Draper & Smith, 2014). The within-subject study design of

Experiment 2, where each child played with two toys, resulted in observations that

were dependent on each other. This is largely a study design issue. Separate linear

regression models were created for data collected from children’s interactions with

the two directly manipulated interfaces.

Multiple regression analysis requires that the data should not show

multicollinearity (Draper & Smith, 2014), which occurs when two or more predictors

are highly correlated. Since there were more than three predictors, Variation Inflation

Factor (VIF) was considered more reliable than pairwise correlations to determine

multicollinearity in the data (Kutner, Nachtsheim, & Neter, 2004). VIF estimates

inflation in a regression coefficient for a predictor because of the linear dependence

on other predictors.

VIF values have a lower bound of 1, and no upper bound. A value of 1 means

there is no correlation between the predictors in regression analysis. VIF values

between 1 and 5 indicate little correlation between the independent

variables/predictors. VIF values between 5 and 10 mean there is moderate

correlation, and values greater than 10 mean there is a very high correlation between

the predictors.

Cohen et al. (2013, p. 419) suggested the use of ridge regressions to tackle

multicollinearity issues. Multicollinearity results in least square estimates that are

unbiased, and large variances that might be far from the true value. Ridge regression

adds a degree of bias to the regression estimates, thus reducing the standard errors

and resulting in reliable regression estimates. Separate ridge regression models were

created for data collected from children’s interactions with the two directly


manipulated interfaces used in Experiment 2. The regression coefficients, t-values,

and significance values were then compared with the results from the linear

regression results to determine the impact of multicollinearity on the results. STATA

statistical software was used to run Ridge regressions, as SPSS does not support it.

Ridge regressions generated robust standard errors, with no significant

differences between the linear regression and ridge regression results in terms of the

regression coefficients, t-values, and significance. As a result, it was concluded that

although the linear regression models showed multicollinearity, it had no appreciable

effects on the linear regression statistics. Thus, the statistics derived from the linear

regression model were used to discuss the results of the study (Chapter 7).

5.4.4 Integration of findings

Integration of the findings of this research was carried out on two levels: the

first at the experimental level, where findings from the qualitative and quantitative

analyses were integrated and reported; the second, where findings from Experiment 1

and Experiment 2 were integrated.

Triangulation was used to integrate qualitative and quantitative findings at the

experimental level. Erzberger & Kelle (2003) used the term ‘triangulation’ as a

metaphor to represent the process of integration, linking theoretical propositions and

the research findings. Considering the deductive nature of its analytical approach, the

triangulation representation of the integration process was deemed suitable for this

study. It has provided a greater understanding of the links between theory and

empirical findings, and aided the development of an additional theoretical

proposition. The triangulation involved assessing the logical relationships between

theoretical propositions and the findings.

The findings and theoretical propositions in Experiment 1 and Experiment 2

were collated to represent the overall findings in a Model for Embodied Intuitive

Interaction (MEII) (discussed in Section 8.2). The focus of this study was on the

design of interactions, rather than the design of products. Thus, the outcome is MEII:

an interaction model that will facilitate the design of products with Embodied

intuitive interactions. The findings from Experiment 1 and Experiment 2 were

integrated into the continuum of intuitive interaction (Blackler, 2008) resulting in an

Enhanced Framework for intuitive interaction (EFII) (discussed in Section 8.1.2).


Buur & Andreasen (1989) described design models as representations of the

properties of objects that designers create. These representations provide insights into

the properties of the product that is being designed. Interaction models also

represent ways in which interactions with certain properties such as intuitiveness

(Blackler & Popovic, 2016), Embodiment (Dourish, 2001), thoughtfulness (Löwgren

& Stolterman, 2004), and seductiveness can result in the design of playful and

effective user experiences (Anderson, 2011). Interaction models take various forms

such as mathematical formulae, verbal descriptions, sketches, block diagrams, and

functional models, depending on the expertise of the designer and the level of

abstraction of the design process. An interaction model was developed in this

research study and the model for Embodied intuitive interaction (MEII) is presented

in the form of a block diagram (and discussed in Section 8.2).

5.5 SUMMARY

Chapter 5 has described the research design and methods used to collect and

analyse the data. It has also discussed the rationale behind the use of the methods and

techniques used for data collection and analysis for the research topic and user group

under investigation.

Two empirical experiments were conducted. Experiment 1 investigated

intuitive interaction with a physical product and an equivalent virtual interface, and

studied aspects of Embodiment in both of these artefacts. Experiment 2 studied the

impact of each of the aspects of Embodiment on intuitive interaction with a physical

product and a TEI. The experiments were conducted at a local Brisbane (Australia)

state school, and in QUT’s PAS Lab.

Qualitative data in the form of audio and video recordings of observations and

retrospective interviews were collected. The data was analysed using qualitative

thematic analysis, followed by quantitative statistical analysis. The findings were

triangulated with the theoretical proposition (derived from the literature review). The

chapter has also explained how the study’s findings and outcomes were integrated

into MEII, an interaction model (discussed in detail in Chapter 7).

Chapter 6 now describes Experiment 1, and is then followed by a description of

Experiment 2 in Chapter 7. The research methodology unique to each experiment is


briefly discussed, and the results and findings from the experiments are explained

and discussed.


Chapter 6: Embodied Intuitive Interaction: Physical and Virtual

A comparative empirical study was conducted to firstly investigate researchers’

previously un-tested beliefs and claims that physical Jenga (Hasbro & Scott, 2001;

Scott, 2006) is more intuitive than virtual Jenga (app) (Natural Motions & Scott,

2011). The two systems, one physical product (physical Jenga) and an equivalent

virtual interface (Jenga app), were investigated for aspects of Embodiment that

facilitate children’s intuitive interaction. The two interfaces represent the extreme

ends of the physical-virtual continuum (Figure 1). The observational and

retrospective interview data were analysed for intuitive interaction and Embodiment.

This chapter addresses the following research sub-question:

- Can Embodiment facilitate intuitive interaction in children?

This chapter begins with a description of the research methodology and the two

toys that were used in the study (Section 6.1). The selection and recruitment of

participants is also briefly explained in this section. Section 6.2 then describes the

thematic and quantitative analysis of the data collected. Results from the analysis are

presented in Section 6.2.2 with the results discussed in Section 6.3.

6.1 METHODOLOGY

A between-subjects study was conducted in order to investigate differences in

intuitive interaction with a physical Jenga and a virtual Jenga, and to determine the

aspects of Embodiment that facilitate children’s intuitive interaction. Participants

were divided into two groups: one group played with the physical product—a

physical version of the game of Jenga; the other with a virtual interface—Jenga app.

The allocation of children to a particular group was random to make sure that each

child had an equal chance of being assigned to either group. The objective was to

observe children playing with a toy and an equivalent, similar looking app, with the

same set of game rules applied in each case. Type of Toy was the independent

categorical variable for the experiment, with two levels—physical and virtual—that

were not intrinsically ordered.


6.1.1 Game Description

A physical version of the Natural Motions’ Jenga toy was chosen as the

physical product, to be compared with its equivalent virtual version in the form of an

app ( Figure 8). Both games had similar rules. The physical Jenga toy consisted of

54 wooden blocks. The game was set up by stacking all blocks in 18 layers of three,

placed next to each other lengthwise, and perpendicular to the layer below.

The virtual interface was an Android app on a tablet that had exactly the same

game setup of 18 layers of blocks. The Jenga app uses real-time 3D physics

simulation, and recreates the behaviour of the physical Jenga toy. Each block is

affected by the surrounding blocks just as it would be in the real world, and recreates

the same strategic depth as the physical toy. It can be played in both single and

multiplayer options; however, the multiplayer option was chosen for this study.

Figure 8 Physical Jenga toy (left), and virtual Jenga app (right)

Blocks can be removed from the stack by swiping and/or tapping on them.

Once a block is removed from the stack, it is automatically taken to the top of the

stack for placement, without any player intervention. A tap on the block places the

block on the stack. The actions required to remove and stack a block are shown in

Figure 9.


Swipe block sidewards

Swipe block outwards

Tap on the block

Block positioned on top of the

stack for placement

Tap on the block initiates

placement

Block is dragged into the

position on the stack

Figure 9 Swiping and tapping to remove the block (above); placing the block on the top of the stack (below)

The lack of materiality in the app is compensated for through deliberate clues

such as warning signs in the form of white, pink, and red coding on the blocks when

touched or tapped, as shown in Figure 10 - (a) A white-coded block suggests that it is

safe to remove the block; (b) a pink-coded block suggests that it is slightly unsafe to

remove the block; and (c) a red-coded block suggests that it is highly dangerous to

remove the block. The stack is rotated by swiping left or right anywhere on the

screen, but not on the stack itself.

(a) (b) (c)

Figure 10 Colour-coded warnings of the danger of crashing the stack


6.1.2 Participants

One hundred and eight children in the 5–12 year age group were randomly

selected from a pool of volunteers recruited through a local primary school. 56

children (28 pairs) played with physical Jenga, and 52 children (26 pairs) played with

virtual Jenga.

Jenga being a popular game, children could be more familiar to physical Jenga

than the virtual Jenga. Blackler et al.'s (2010) investigation into intuitive interaction

showed that more familiar interface features are used more intuitively. This means

that familiarity of physical Jenga with children could affect intuitive interaction

results in the Experiment. However, familiarity of physical Jenga in children

represents experience and familiarity of children to physical objects, physical

interactions and material properties. Thus, the familiarity and popularity of physical

Jenga could be attributed to intuitiveness of the toy. The objective of Experiment 1

was to study what makes certain interfaces more intuitive for children’s interaction.

Children played together in a team which ensured that their natural play was

not affected by the possible intimidation of being with children and a researcher

whom they did not know. Parents were not present during the study, as children

behave differently when around adults, especially parents and teachers (Gardner,

2000).

6.1.3 Setting and Procedure

Observations and retrospective interviews and co-discovery verbal protocols

were used to collect the data. Two children were paired to play against the

researcher. All three (two children and the researcher) took turns to play. Each turn

involved taking one block out from any layer of the stack, with the exception of the

layer just below an incomplete top layer above it, and placing it on the topmost layer.

The game ended when the stack fell completely, or if any block fell from a stack.

The team responsible for the stack falling lost the game.

Experiments were conducted in a classroom at a local primary school and

QUT’s PAS Lab. Children were welcomed into the room and made comfortable and

at ease by playing some warm up games such as Blokus (Mattel, Sekkoïa, &

Tavitian, 2000). Children were then introduced to the toy/app. Studies on intuitive

interaction are based on the premise that intuitive interaction relies on prior


experience and knowledge (Blackler et al., 2010). Thus, telling participants how the

products and interfaces work could skew the application of past experience and

knowledge of the participants. Thus, children were not explained anything about how

toys and games worked. The children were told that the objective of the game is to

stack the blocks one above the other without letting the stack to fall over. The blocks

in each layer of the stack are placed perpendicular to the blocks in the lower layer.

The children were told that they would be playing together in a team against the

researcher, and everyone takes turn one after another. The winning team would win a

prize from the ‘goodie box’. The ‘goodie box’ (funded by Creative Industries Higher

Degree Research grants and the Design Research Society) was a box filled with

children’s toys and games. The same instructions were given to both the groups of

children, those playing with the physical toy and those playing with the virtual app.

Children were asked to set up the game wherever they wanted to do so. Some

children opted to set up the physical game on the table, and others opted to play on

the floor (Figure 11). All children opted to play with the virtual app on the table

(Figure 12). This facilitated creation of a natural setting for their playtime.

The entire game play was video recorded for analysis, using two digital video

cameras. For the experiments with the physical toy, one camera was placed in front

of the children and the other on the side, to capture their interactions and facial

expressions from all possible angles during their play (Figure 11). For the

experiments with the virtual interface, one camera was placed in front of the children

to capture facial expressions, and the other behind them to focus on their hands as

they interacted with the tablet (Figure 12).

The game play was followed with a retrospective interview where the children

were shown the video of their game play and asked to talk about it. Two cameras

were used to record the retrospective interviews. One camera captured the screen that

the children were watching together and the second camera captured the face of the

children. They did this, and conversed with each other as they watched. Co-discovery

verbal conversations captured during the game play and retrospective interviews, and

the video recordings, were then used to code the interactions of the children with the

systems. The entire session lasted for 45 minutes to one hour.


Figure 11 Children playing with the physical toy

Figure 12 Children playing with the virtual interface

6.2 ANALYSIS

The video recordings of the game play and retrospective interviews were

analysed using a two-step process, as described in Section 5.4. A qualitative thematic

analysis in the form of coding was followed by a quantitative analysis of the coded

data.

Audio-visual recordings of the game play and retrospective interviews were

coded concurrently in Noldus Observer XT software (Version 12.5). All the four

videos (that is, front and back camera recordings of the observations and the two

recordings of the retrospective interviews) were synchronised in Observer to the

same event marker in all four videos (Figure 13). An event marker was set during the

observations with a prompt of ‘let us start’ which was also captured during the replay

of the observations in the retrospective interviews.


Figure 13 Coding environment in Observer XT

The entire game play with Jenga (both the toy and the app) was divided into

three main behaviours and interactions: Decide, Remove and Stack. Decide

represented behaviour of the children where they were deciding which block to

remove and how to remove from the stack. Remove represented actual removal of the

block and Stack represented placing the removed block on the stack. Each of these

behaviour streams were parsed into meaningful actions which were then coded for

themes within two theme groups: Type of interaction and Aspects of Embodiment.

The coding heuristics for Type of interaction and Aspects of Embodiment are given in

Appendix D and Appendix E. Rigour in the determination of heuristics in the context

of Experiment 1 was ensured through the detailed observations of all four recordings

(front and back cameras during game play, and retrospective interviews).

The first level of codes for each of the theme groups, referred to as ‘themes’,

corresponding to Types of interaction were: Intuitive, Non-intuitive, and Partially-

intuitive; while the themes relating to Aspects of Embodiment were: Physical

affordances, Perceived affordances, Emergence, Scaffolding, and Co-operative

activity. The data were coded for each of the children separately. The interactions


and behaviour of the children with the physical and virtual Jenga were coded for

Types of interaction and Aspects of Embodiment.

Data were coded with caution: every observation was checked twice and, at

times, thrice. All coding was done by one researcher and, to avoid observer bias, data

were coded twice, with a break of 15 days between the two coding sessions.

Reliability analysis was carried out in SPSS to determine the intra-rater reliability.

ICC estimates and their 95% confident intervals were calculated using SPSS

statistical package version 23 (SPSS Statistics, 2016) based on a single measurement,

absolute-agreement, 2-way mixed-effects model. The ICC results are as follows:

ICC 3,1 0.89, 95%ConfidentIntervalof0.83 0.95

Based on the guidelines from (Koo & Li, 2016) (Appendix A), it can be

concluded from the ICC results that the intra-rater reliability was ‘good’ to

‘excellent’.

All the coded data were then exported to Excel, where the numbers of codes or

themes were totalled for each child. The totalled codes were used as dependent

variables (DVs): Number of intuitive interactions, Number of physical affordances,

Number of perceived affordances, Number of emergence events, Number of

scaffolding and Number of co-operative activity. The absolute and relative time of

each of the coded events were also reported by Observer in the exported data. The

time spent by each child deciding which block to remove and how to remove (Decide

behaviour) was evaluated from the exported time information. This time was also

totalled for each child for the entire episode of play and used as the DV, Latency to

decide.

6.2.1 Thematic Qualitative analysis

Types of interaction – intuitive, non-intuitive and partially intuitive

Intuitive interaction involves utilising knowledge gained through other

experience(s), is fast, and generally unconscious (Blackler, Popovic, & Mahar,

2010). The coding heuristics employed to code for the intuitive interactions are

derivations of methods developed by (Blackler, 2008)

Children were considered to be reasoning unconsciously when they could not

explain why they chose a certain block or how they removed and/or stacked the

block. One of the participants, when asked how he chose the block for removal, said,


“I don’t know. I just did it”

Another participant chose a block to remove after tapping at the blocks looking

for a loose block. This participant when asked the same question, said,

“I removed this block because it will balance the stack”

Although the participant did explain why he chose the block, the verbalisation

did not match his action.

Intuitive interaction is associated with high degrees of certainty, confidence

and expectation with respect to correct use of a feature (Blackler, 2008). There is no

correct or incorrect way of playing with a toy. Thus, when participants were certain

and confident that the stack would not fall because of their choice of block for

removal or during the removal and stacking of the block, the behaviour was Intuitive

Interaction. One participant, while describing how he removed the block, said

“I know the stack will not fall because it is balanced.”

The above conversation between children not only shows that the child was

certain and confident of her decision but also was reasoning unconsciously because

she was unable to verbalise the actual reason and then gave a metaphorical reason,

being in balance is equivalent to not falling.

Intuitive interaction is associated with faster decision making. Blackler, (2008)

coded correct use of a feature with not more than 5 seconds of hesitation as Intuitive

Interaction. Since play is not associated with correct use, latency was measured as

time taken to decide irrespective of whether that decision results in a win or a loss.

When a participant made a decision within 5 seconds (Blackler, 2008) and when the

decision was made with a degree of certainty and unconscious reasoning, the

behaviour was coded as Intuitive Interaction.

Partial intuitive interaction was coded when children showed signs of Intuitive

Interaction as well as Non-Intuitive Interaction. One participant clearly verbalised his

behaviour (Non-Intuitive Interaction) but was certain and confident about his

decision (Intuitive Interaction),

“I picked a loose block so that it easily comes out…It will not fall for sure.”


Aspects of Embodiment

Physical Affordances - Children used natural cues such as visual cues from

the spatial orientation of the blocks and the stack and haptic or tactile clues from the

material properties of the blocks to decide which block to remove. Children checked

whether the blocks were loose enough to be removed from the stack easily by

tapping on the blocks. Side blocks were removed from the stack by sliding the block

out of the stack. The middle blocks were removed by either pushing or pulling the

block out of the stack. Children placed the block either in the middle or the side on

the top of the stack depending on the spatial structure of the stack. They used

contextual cues such as tapping on the blocks to determine a suitable block to

remove from the stack.

Perceived Affordances –Children used their past experience with physical

blocks and their knowledge about physical and material properties such as mass,

rigidity, mobility, etc. Children often preferred to remove a middle block as they had

learned from their past experiences with everyday artefacts that as long as the side

blocks are kept intact the stack will remain balanced and will not fall. One of the

children explained his decision to remove only middle blocks as

“…I want to create windows in tall buildings...Tall buildings don’t fall, do

they?”

One of the children while explaining why he/she kept the blocks at the lower

end of the stack intact, said,

“…The base has to be balanced just like a chair is balanced on 4 legs…..”

When children could not directly use their experience from physical world,

they used population stereotypes from everyday life to interpret deliberate clues in

the Jenga app, for example they interpreted red colour codes on the blocks in the

virtual toy as highly risky to remove the block and white colour codes as safe to

remove the block. Children used their past experience and knowledge using digital

technology such as tablets. Children swiped at the tablet screen to remove the blocks

and to turn the stack around.

Children used deliberate clues such as colour codes, warning signs and

symbols, arrows to bring users’ attention to a feature etc. to make connections with

their prior knowledge and experience.


Emergence involves use of features in a product that result in change in

strategies to interact with the product and the interface. Children would often start

the game with the removal of middle blocks from the stack as they had learnt from

their past experience that it would keep the stack balanced. As the spatial structure of

the stack evolved over a number of turns, the children started removing the side

blocks. Similar changes in game strategy were also observed during a turn, such as

when children sensed that a block was too stiff to remove, and they started removing

another block.

Scaffolding was evident when children used different types of scaffolds in the

form of processes and support mechanisms while playing. The most effective

scaffolds used by children are people around them (Sharifnia et al., 2015). Imitating

strategies is one such scaffolding process; children learned different strategies from

other players, such as push and pull technique to remove the block from the stack

and turning the stack around by swiping left-right on the tablet screen.

Children offered physical support by holding the stack while the other child

removed the block. Children demonstrated use of physical space to assist them to

decide which block to remove and the actual removal of the block. Children

inspected the stack from all possible angles and moved around the tangible stack to

decide and remove the block from the stack in a tangible toy. This allowed them to

gain visual cues from the stack, which resulted in successful removal of a block from

the stack.

Co-operative activity between children was evident when they worked

together physically and verbally to develop strategies for the game. Children co-

operated through verbal encounters to develop highly organised game strategies.

Children discussed their strategies while playing with the toy. Some discussions were

brief and simple such as,

“Just keep removing the middle blocks…”

While others were more complex,

“Let’s remove all the side blocks from the top, keep the blocks at the bottom

intact…”

Actions such as pointing or touching at the blocks/stack, pulling a block out

from the stack on the other player’s turn and then either handing it to the other player


or stacking the block on the top of the stack, etc. are also examples of co-operative

activity demonstrated by children in the game play.


The coded data with the totalled codes/themes were analysed quantitatively in

SPSS. The physical Jenga and the virtual Jenga were compared for intuitive

interaction, and for aspects of Embodiment that facilitated intuitive interaction in

both the physical and virtual toy. The variables (dependent and independent) chosen

for quantitative analysis are shown in Table 4.

Table 4 Dependent for measures of intuitive interaction, successfulness and aspects of Embodiment

The DVs Number of intuitive interactions, Number of physical affordances,

Number of perceived affordances, Number of emergence events, Number of

scaffolding and Number of co-operative activity were taken directly from the totalled

codes/themes in Excel. As the game progressed, the stack grew taller as more blocks

were removed and added to the stack. The successfulness of the game was measured

by noting the number of layers added over and above the 18 layers that the children

and the researcher started playing the game with. The DV Number of layers added

was noted for each child during the experiment, and double checked during video

analysis. The time taken to decide which block to remove and how to remove from

the stack, totalled for the entire episode of the game for each child was used as DV

Latency to decide.

Independent Variable Dependent Variables

Type of Toy – Physical and Virtual Measures of Intuitive Interaction

Number of intuitive interactions

Latency to decide

Measure of Successfulness

Number of layers added

Measures of aspects of Embodiment

Number of Physical affordances

Number of Perceived affordances

Number of Emergence events

Number of Scaffolding

Number of Co-operative activity


Intuitive interaction in physical and virtual interfaces

The descriptive statistics for the Number of intuitive interactions, Number of

layers added and Latency to decide corresponding to the Independent Variable (IV) -

Type of Toy: Physical Jenga and Virtual Jenga are presented in Figure 14 and Table

5.

Figure 14 Comparison of Number of Intuitive Interactions; Number of Layers Added; and Latency to decide for physical and virtual Jenga

Table 5 Descriptive statistics for Number of intuitive interactions, Number of layers added and

Latency to decide corresponding to Type of Toy: Physical and Virtual Jenga

Descriptive (→) Statistics DVs (↓)

Type of Toy Physical Jenga Virtual Jenga Mean Median Standard

Deviation Mean Median Standard

Deviation Number of intuitive interactions

26.59 24 13.84 10.16 9.5 7.09

Number of layers added 8.90 7.5 4.46 2.94 2.5 2.43 Latency to decide (secs) 10.83 7.47 13.14 13.83 10.12 13.16


A Mann-Whitney U test was run to determine if the differences in Number of

Intuitive Interactions, Number of Layers Added, and Latency to decide between

physical Jenga and the virtual Jenga were statistically significant. The Mann-

Whitney U test works by ranking each score of the dependent variables, irrespective

of the group it is in (physical Jenga or virtual Jenga), according to its value, with the

smallest rank assigned to the smallest value. The ranks obtained for each of the

groups are then averaged. This results in a mean rank for each of the groups -

physical Jenga and virtual Jenga (shown in Table 6).

The null hypothesis of the Mann-Whitney U test is that the mean rank is the

same for both the groups, physical Jenga and virtual Jenga. However, if one group

tends to have higher values than the other group, that group's scores will have been

assigned higher ranks and will have a higher mean rank (and vice versa for the group

with lower scores). It is this difference in mean rank that is tested by the Mann-

Whitney U test for statistical significance. This approach was used to determine

whether the differences in the DVs for physical Jenga and virtual Jenga are

statistically significant. The mean ranks obtained for the physical Jenga and the

virtual Jenga are shown in Table 6.

Table 6 Mean Rank Values of Number of Intuitive Interactions, Number of Layers Added, and Latency

to decide for each type of toy

The Mann Whitney U Test statistic of the dependent variables is presented in

Table 7.

Table 7 Mann-Whitney U Test Statistic of Number of Intuitive Interactions, Number of Layers Added,

and Latency to decide for each type of toy

DVs (→) Type of Toy (↓)

Mean Rank Values Number of Intuitive Interactions

Number of Layers Added

Latency to decide

Physical 74.26 76.52 49.82 Virtual 33.22 30.79 59.54

Number of Intuitive Interactions

Number of Layers Added

Latency to decide

Mann-Whitney U 2562.5 2689 1194 Standardised test statistic (Z) 6.809 7.613 -1.611 Asymp. Sig. (2-tailed) (p) 0 0 0.107 Effect Size (d) 0.66 0.73 0.16


Putting all the results together (both descriptive and inferential), Number of

Intuitive Interactions for the physical Jenga (mean = 26.59, median = 24, std.

deviation = 13.84) was statistically significantly higher than for the virtual Jenga

(mean = 10.16, median = 9.5, std. deviation = 7.09), U = 2562.5, p ≈ 0 (p<0.05),

d=0.66.

Number of Layers Added for the physical Jenga (mean = 8.90, median = 7.5,

std. deviation = 4.46) was statistically significantly higher than for the virtual Jenga

(mean = 2.94, median = 2.5, std. deviation = 2.43), U = 2689, p ≈ 0 (p<0.05), d=0.73.

Effect size (d) indicates that the differences in Number of Intuitive Interactions and

Number of Layers Added between the physical Jenga and the virtual Jenga are strong.

Latency to decide was not statistically significantly different between children

playing with the physical Jenga (mean = 10.83, median = 7.47, std. deviation =

13.14) and those playing with the virtual Jenga (mean = 13.83, median = 10.12, std.

deviation = 13.16), U = 1194, p = 0.090 (p>0.05), d=0.107. Effect size (d) = 0.107,

indicates that the difference in Latency to decide between the two groups is trivial.

The above results suggest that physical Jenga possess properties and features

that make them more intuitive than the virtual Jenga. The latency parameter is

insignificant in both the systems. This could be due to the context of play where

children are involved in pragmatic actions to reach the ultimate goal specific to a

game, and also in epistemic actions that allow them to offload tasks from their mind

to actions.

Aspects of Embodiment in physical toy and virtual interface

The two toys, physical Jenga and virtual Jenga, were then analysed for use of

aspects of Embodiment to determine the extent to which each of the aspects of

Embodiment contribute to intuitive interaction. The descriptive statistics for the use

of aspects of Embodiment corresponding to the Independent Variable (IV) - Type of

Toy: physical Jenga and virtual Jenga is presented in Table 8 and Figure 15.


Table 8 Descriptive statistics for Number of uses of aspects of Embodiment for physical and virtual

Jenga

Figure 15 Box plot of use of aspects of Embodiment in physical and virtual toys

The physical Jenga demonstrated the highest uses of physical affordances

(mean = 22.09, median = 20.0, std. dev. = 12.28), followed by perceived affordances

(mean = 6.97, median = 6.0, std. dev. = 5.29). This was followed by use of

Descriptive (→) Statistics DVs (↓)

Type of Toy Physical Jenga Virtual Jenga Mean Median Standard

Deviation Mean Median Standard

Deviation

physical affordances

22.09 20.0 12.28 5.79 4.0 4.98

perceived affordances

6.97 6.0 5.29 7.94 7.5 6.39

Emergence 4.25 3.0 3.20 0.75 0.0 1.31 Scaffolding 3.161 2.0 2.90 3.16 3.0 2.04 Co-operative activity

5.64 4.0 4.76 1.90 1.0 2.01


cooperative activity (mean = 5.64, median = 4.0, std. dev. = 4.76), emergence (mean

= 4.25, median = 3.0, std. dev. = 3.20), and scaffolding (mean = 3.161, median = 2.0,

std. dev. = 2.90). Physical affordances were the prime contributor to intuitive

interaction with the physical Jenga.

The virtual Jenga demonstrated the highest uses of perceived affordances

(mean = 7.94, median = 7.5, std. dev. = 6.39), followed by physical affordances

(mean = 5.79, median = 4.0, std. dev. = 4.98). This was followed by uses of

scaffolding (mean = 3.16, median = 3.0, std. dev. = 2.01), cooperative activity (mean

= 1.904, median = 1.0, std. dev. = 2.01) and emergence (mean = 0.750, median = 0.0,

std. dev. = 1.31). Perceived affordances were the prime contributor to interaction

with the virtual Jenga.

A related Samples Friedman test was run to determine if the differences in uses

of aspects of Embodiment in each of the toys, were statistically significant. Related

samples Friedman test works in a similar way to Mann Whitney U test except that it

is used to determine significance within samples (within physical Jenga and virtual

Jenga). The mean ranks obtained for the uses of each aspect are shown in Figure 16.

The use of aspects of Embodiment were statistically significantly different for

interactions with the physical Jenga, χ2(4)= 119.5, p <0.05. Use of aspects of

Embodiment were statistically significantly different for interactions with the virtual

Jenga, χ2(4)= 108.77, p <0.05.

Pairwise comparisons were performed for statistical significance, with a

Bonferroni correction for multiple comparisons. The results for the physical Jenga

are shown in Figure 16 (left), and for the virtual Jenga in Figure 16 (right). The mean

rank values for each of the aspects of Embodiment for interactions with the toys are

presented. The black thick lines represent statistically significant differences between

pairs, while the grey thin lines represent statistically insignificant differences.

Pairwise comparisons for the physical toy indicate that the difference between

physical affordances and other aspects is statistically significant. On the other hand,

pairwise comparisons for the virtual toy indicate that the difference between physical

affordances and other aspects is statistically significant, with the exception of

perceived affordances. Perceived affordances also demonstrated statistically

significant differences with the other aspects, with the exception of physical

affordances.


Figure 16 Pair-wise comparisons of aspects of Embodiment for interactions with physical toy (left) and virtual toy (right) with mean ranks for each aspect

6.3 DISCUSSION

The findings suggest that the physical toy was more intuitive, and supported

more successful game play (in terms of layers added) than the virtual app. This study

also investigated the aspects of Embodiment that make physical Jenga more intuitive

than virtual Jenga. The results showed that the intuitive use of aspects of

Embodiment was more evident in the physical Jenga than the virtual Jenga. Physical

affordance was the main facilitator of intuitive interaction in the physical Jenga,

followed by perceived affordances, co-operative activity, scaffolding, and

emergence. Perceived affordances were the main facilitator of intuitive interaction in

the virtual toy. However, there was no significant difference in intuitive use of

perceived affordances between physical Jenga and virtual Jenga.

Children used physical affordances and their experiential knowledge of playing

with blocks, and other similar toys and games, to play with the physical toy. They

also used their experiential knowledge from everyday life such as seeing buildings

with windows, or knowing that a broader base keeps the stack more stable. Spatial

orientation of the stack and the blocks, and material properties of the blocks (such as

their amount of stiffness) provided assistance in playing the game. Children used

visual, material, spatial and contextual natural clues from the blocks and the stack in

their decision making before removing blocks from the stack. The spatial layout of

the blocks in the physical toy offered the visual natural clues to tap on the block.

However, when they sensed that the block was not loose, some used two hands to


remove the block so that the stack did not fall down, while others looked for another

loose block and continued tapping at the blocks.

Children predominantly used perceived affordances to play with the virtual

Jenga, using their previous experience from the digital world and the physical world

to play with it. The app offered visual cues in the form of the three-dimensional

spatial orientation of the block on the tablet. The app also mimicked the wobbliness

of a physical stack in the real world. The game designers represented the materiality

of the physical Jenga toy in the app by contextual cues in the form of white, red, and

pink colour codes on the blocks, and crash signs to warn players of the danger of

stack collapse. However, the children did not often detect these contextual perceived

affordances. Some were able to learn them as they played through a number of turns.

However, most toppled the stack before they had any opportunity to notice the cues,

or gave up playing the game. Thus, although attempts were made to incorporate the

physical affordances of physical blocks into the app in the form of deliberate visual

clues, children did not have the relevant knowledge (i.e. sensorimotor or

experiential) to use those features. Thus, there were higher numbers of uses of

physical affordances in intuitive interaction of the physical toy than the virtual app.

However, there was no significant difference between the toys in terms of the use of

perceived affordances for intuitive interaction.

Children were interacting with two interfaces while playing with the virtual

app. Their experiential knowledge of the tablet sometimes contradicted the

experiential knowledge associated with the stack in the app. They had learnt in the

physical world that the blocks had to be pushed or pulled to remove them from the

stack. However, their experience in the virtual world had taught them that the only

way to interact with a tablet (or any other touch screen interface) is to swipe, tap, and

hold onto the screen. Thus, some of the features of the app, such as rotating the stack

by swiping it left and right, were not discovered.

Children often work in teams and groups at school. Peer support mechanisms

are often used in a classroom setting, where children offer support to others in the

class and, at the same time, gain support from others (Berk & Winsler, 1995;

Hammond & Carpendale, 2015). Children offered physical support by holding the

physical stack in the physical toy together while the other child removed and stacked

the block. Some children unknowingly imitated each other’s game strategies while


playing with the physical toy, while there was limited evidence of such processes

being used with the virtual app. High levels of physical support, imitation of game

strategies, and use of the space around them to interact with the physical toy, explain

the higher numbers of intuitive uses of scaffolds with the physical toy than with the

virtual toy.

Children played with the physical toy together, even though they were asked to

take turns to play. They discussed strategies and provided verbal suggestions to each

other while playing. The material and spatial properties offered by the physical toy

allowed the display of real time information about the object (Sellen & Harper,

2002), and provided a mechanism for efficient co-ordination of information between

the players (Vyas et al., 2012). Children intervened in the other player’s game, for

example, to change the placement of the block on the top of the stack so that the

stack remained balanced, or to complete the removal of the block from the stack. On

the other hand, when playing with the virtual app, they waited for the other child to

finish interacting, and avoided intervening in their play. This was the result of their

perception that two people or more cannot interact with a tablet at the same time.

Thus, the physical toy demonstrated higher numbers of intuitive uses of the co-

operative activity aspect than the virtual app.

The effective use of visual, material and contextual natural clues in the

physical toy resulted in changes to its spatial and material properties (stiffness and

looseness of the blocks in the stack, for example); this, at times, resulted in changes

in strategies, previously explained as ‘emergence’. For example, removal of a block

from the stack resulted in changes in system behaviour (e.g. a wobbly stack and stiff

and loose blocks), and changes in system configuration. This resulted in emergent

interaction as the players had to change their strategy to remove the blocks from the

stack.

The lower the amount of prescribed behaviour built into the system, the higher

the degree of emergence (Pfeifer et al. 2005). The simplicity of the blocks, which

have a loosely defined behaviour, allowed emergence. On the other hand, the virtual

app, with its pre-defined, pre-programmed behaviour, inhibited emergent

interactions. Thus, the intuitive use of the emergence aspect of Embodiment was

more evident in the physical toy than in the virtual app.


Intuitive use is fast and, thus, it was expected that children would take longer to

decide which block to remove when playing with the virtual interface compared with

the physical toy (Blackler, 2008). However, there was a trivial difference in Latency

to Decide for both toys. Children were neither pushed to commit to a strategy, nor

were they given a time limit to finish the game. They explored the environment and

socially interacted with the other players. One child, while deciding which block to

remove from the stack, talked about a uniform free day at school:

Child 1: “….why are you wearing yellow socks?”

Child 2: “…we can wear any colour of clothes but not uniform.”

Child 1: “…but why are you wearing yellow?”

Children demonstrated social and exploratory behaviour with both physical toy

and virtual app, and this affected the Latency to Decide dependent variable. This

suggests that no time measure is relevant when investigating children’s play, unless

they have been told to finish as quickly as possible.

6.4 SUMMARY

This chapter has presented a comparative study of physical Jenga and virtual

Jenga for intuitive interaction, and the use of aspects of Embodiment in intuitive

interaction with these toys. The results showed that physical Jenga demonstrates

more intuitive interactions than virtual Jenga, confirming previously un-tested claims

of researchers that a physical product is more intuitive than its virtual counterpart

(Hornecker & Buur, 2006; IKindsmuller, et al., 2009; Mihajlov et al., 2015; Olson et

al., 2011; Sapounidis et al., 2015; Seo & Lee, 2013). Children showed high levels of

uses of physical affordances, emergence, scaffolding, and co-operative activity in

interaction with the physical toy in comparison to the virtual toy. They showed the

equal likelihood of using perceived affordances in interaction with the physical and

the virtual toy.

The interaction with virtual systems relies solely on experiential learning

during the use of the product, or from the prior use of similar products. However,

children can either abandon their use of the product, or fail to meet given goals

before they have detected and learnt the deliberate clues embedded in the design.


Thus, a design that is solely based on experiential knowledge and perceived

affordance does not ensure effective interaction.

Chapter 7 now describes the second experiment that was carried out to

determine how aspects of Embodiment can facilitate children’s intuitive interaction.

Two systems that allow direct manipulation of physical elements, a physical toy and

a TEI with physical and virtual elements, were investigated for intuitive interaction.

They were then analysed for aspects of Embodiment that have a significant impact

on children’s intuitive interaction.


Chapter 7: Primary Predictors of Embodied Intuitive Interaction

Experiment 1 described a comparative study of a directly manipulated physical

Jenga and virtual Jenga for intuitive interaction and, more specifically, for aspects of

Embodiment that facilitate intuitive interaction in these systems. Experiment 1

confirmed previously un-tested claims in the literature that physical products are

more intuitive than their virtual counterparts. The study further explained the role of

aspects of Embodiment in intuitive interaction with physical Jenga and virtual Jenga.

Children showed higher levels of the use of physical affordances, emergence,

scaffolding, and co-operative activity in intuitive interaction with physical Jenga than

with virtual Jenga. The use of perceived affordances is equally likely in intuitive

interaction with physical Jenga and virtual Jenga. The findings suggest that the

presence of these aspects of Embodiment in the design of a system could ensure

children’s intuitive interaction.

Given that the directly manipulated physical products have sufficient aspects of

Embodiment to allow children to interact with them intuitively, Experiment 2

analysed two directly manipulated interfaces separately (not compared)—a physical

product, and a Tangible Embodied Embedded Interface (TEI)—for aspects of

Embodiment that have a significant impact on children’s intuitive interaction. The

objective was to study variability of intuitive interaction with respect to the aspects

of Embodiment in a physical product and a product with both physical and virtual

elements, so that the physical-virtual continuum is sufficiently covered in the study.

Experiment 2 answers the following research sub-questions:

- Which design aspects of Embodiment facilitate children’s intuitive

interaction

- To what extent do the design aspects of Embodiment facilitate intuitive

interaction in children?

The two products chosen for this study were Monkey Blocks (Popular

Playthings, 2014) and Osmo (Tangible Play, 2014b). Monkey Blocks is a physical

toy consisting of physical blocks. Osmo is a TEI with a physical and a virtual space


that are distinct and separate. Both the toys have a common interaction mechanism

by way of direct manipulation of physical objects. Choice of these two toys was

influenced by factors such as accessibility to toys, especially a TEI, cost of TEI and

time taken to play with the toys in an experimental setup (children, especially

younger children, get tired playing for a long time).

This chapter describes the methodology of the data collection, and its

subsequent analysis. This is then followed with the results and discussion.

7.1 METHODOLOGY

A within-subjects observational study was carried out with 42 children, aged

5–12 years, at PAS Lab, QUT (Australia). Children were paired with their friends or

siblings. Parents and children were invited to participate in the study during the

school holidays, and were asked to bring a friend or a sibling to play with. The

parents were however asked not to be present during the study. If they insisted on

being around or present for the study, they were requested to sit on the other side of a

two-way mirror. The children and their parents were known to the researchers

through personal contacts, and through their participation in Experiment 1.

Twenty-one pairs of children were observed playing with two toys, Osmo, a

TEI, and Monkey Blocks, a physical product. Monkey Blocks represents the extreme

left of the physical-virtual continuum (Figure 1), while Osmo represents the middle

of the continuum. Both the toys involved direct tactile interaction and manipulation

with physical objects to achieve game-specific goals. Children and parents were first

welcomed into the lab, made comfortable, and then asked to sign the consent forms.

Children were instructed to start playing with one of the toys.

While playing with Osmo, children were instructed to start playing from Level

1, and to progress to other levels after they had completed the previous one. While

playing Monkey Blocks, children were instructed to arrange the blocks by looking at

a black and white pictorial arrangement of their choice on a sheet of paper (Figure

21), and to work together as a team. They were given a maximum 30 minutes to play

each game. However, they were allowed to stop playing before 30 minutes if they

wished so. They played both the games one after another. The order in which a child

pair played each of the games was varied from one observation to the other, ensuring

that there were an equal number of pairs playing each game first. This was to ensure


that the carry-over effects balanced out across the two game plays (Gray & Kinnear,

2012) .

Both the game plays were video recorded for analysis. Two digital video

cameras were used to record the play. One camera was placed in front of the

children, and the other on the side to capture their interaction and facial expressions

during play.

7.1.1 Osmo Game description

Osmo, a TEI game from Tangible Play (Figure 17) , allows physical play with

a virtual system (Ipad). It comes with a reflector, a stand that is attached to an

Ipad,and games that can be downloaded as apps from iTunes.

Figure 17 Osmo setup and Newton app

The game used for the study is called ‘Newton’ (Tangible Play, 2014a) which

was downloaded from iTunes (Apple, 2001) on the iPad (Apple, 2015). The Newton

game consists of 60 levels, each level involving challenges that require manipulation

of objects and drawings placed in front of the screen to guide free falling balls onto

various targets such as spheres on the screen (Figure 18). The challenges require

simple strategies to start with and as children progress through the levels, they are


presented with challenges with bouncing balls, accelerating platforms, teleporters,

and fans.

Figure 18 Children playing Osmo. The view of the tablet screen (on the left) shows Newton game in action and the view of children manipulating objects and drawing in the physical space (on the right)

Children were provided with the following objects to play the game: An A3

paper sheet placed in front of the tablet, two pencils, two pens, two erasers, a cheese

block toy, a ruler, five straws, two lollipops, cardboard strips in both random and

regular shapes (triangle, square, rectangle, and trapezium), a plastic spoon, and two

chop sticks. The direct physical interaction in the game is entirely with the objects in

the physical space, and in the context of achieving the goal of directing balls onto the

targets. The Ipad display screen provides feedback on the manipulation, in relation to

the targets on the screen.

Tangible Embodied Embedded Interfaces (TEIs) were defined and discussed in

Section 1.1. There are several variations of TEIs depending on how the physical and

virtual spaces (in other words, perception and action spaces) are configured. Physical

and virtual spaces could be overlapping (Ullmer & Ishii, 2000) or they could be

distinct and separate (Zuckerman et al., 2016). Osmo and Newton can be considered

as TEIs as they offer a physical and a virtual space for interaction, with integration of

electronics in the form of a tablet (computing element) and a camera. The virtual and

physical spaces are distinct and separate which is in line with the approach taken to

design and develop TEIs by researchers such as Gervais et al. (2016), Mistry & Maes

(2009), Yu et al. (2016), Zuckerman et al. (2016).

Children were asked to play with Osmo and Newton app for as many levels as

they could in a maximum 30 minutes.


7.1.2 Monkey Blocks game description

Monkey Blocks is a logical deduction game, consisting of 12 blocks divided

into three types (4 blocks each) depending on the position of the weights inside the

blocks (Figure 19). Orange blocks are weighted at one end of the block, green blocks

are weighted in the middle, and blue blocks have no weights inside. The game comes

with 12 blocks (4 blocks of each of the three types), and 6 monkeys.

Figure 19 Three types of blocks in Monkey Blocks: orange blocks with weights at one of the ends, green blocks with weights in the middle, blue blocks with no weights.

The object of the game is to stack blocks and monkeys in an arrangement (e.g.

Figure 20) so that they remain balanced, without falling over.

Figure 20 Blocks and monkeys in arrangements

Children were given sheets with 18 arrangements in black and white to use as a

reference. Black and white images were given to prevent children from just copying

the arrangements using the colour codes, but forcing them to rely on physical and


material properties to complete the arrangement. They were asked to build as many

arrangements as they could in maximum 30 minutes. Figure 21 shows an example of

an arrangement in black and white and its equivalent coloured arrangement. All the

18 arrangements given to children in the study and their equivalent coloured images

are in Appendix P.

Figure 21 Example of an arrangement: black and white image given to children (left), equivalent coloured image (right)

7.2 ANALYSIS

The video recordings of the game play were analysed using thematic analysis

(as described in Section 5.4.1) The qualitative thematic analysis (in the form of

coding) was followed by a quantitative analysis of the coded data. Audio-visual

recordings of children playing each of the games were coded as two separate

observations in Noldus Observer XT software (Version 12.5) for each pair of

children. Two videos of the observations, one from the front camera and the other

from the back camera, were synchronised in Observer to the start of the game play.

An event marker was set during the observations with a prompt of ‘let us start’.

The entire game play with Osmo and Monkey Blocks was coded separately for

each child as different observations. The entire episode of game play was

continuously sampled for meaningful interactions which were then coded for themes

within two theme groups: Type of interaction (intuitive, non-intuitive and partially

intuitive) and Aspects of Embodiment (physical affordances, perceived affordances,

emergence, scaffolding and co-operative activity). The coding heuristics for the

theme groups are given in Appendix D and Appendix E. The themes corresponding


to Types of interaction were: Intuitive, Non-intuitive, and Partially-intuitive. The

themes corresponding to Aspects of Embodiment were: Physical affordances,

Perceived affordances, Emergence, Scaffolding, and Co-operative activity. Types of

interaction was configured as mutually exclusive and exhaustive, so that every

behaviour was required to be scored for one of the themes from the theme group.

Aspects of Embodiment was, on the other hand, configured as non-mutually exclusive

and non-exhaustive; in other words, it was not necessary to score every behaviour for

Aspects of Embodiment.

Interactions of each of the children were coded separately. Rigour in the

determination of heuristics in the context of Experiment 2 was incorporated through

detailed observations of the video recordings of game play (using both front and back

cameras). Data were coded with caution: every observation was checked twice and,

at times, thrice. All coding was done by one researcher and, to avoid observer bias,

data were coded twice, with a break of 15 days between the two coding sessions.

Reliability analysis was carried out in SPSS to determine the intra-rater reliability.

ICC estimates and their 95% confident intervals were calculated using SPSS

statistical package version 23 (SPSS Statistics, 2016) based on a single measurement,

absolute-agreement, 2-way mixed-effects model. The ICC results are as follows:

ICC 3,1 0.925, 95%ConfidentIntervalof0.82 0.99

Based on the guidelines from (Koo & Li, 2016) (Appendix A), it can be

concluded from the ICC results that the intra-rater reliability was ‘good’ to

‘excellent’.

All the coded data were then exported to Excel, where the number of codes or

themes were totalled for each child. The totalled codes were used as DVs: Number of

intuitive interactions, Number of non-intuitive interactions, Number of partially

intuitive interactions, Number of physical affordances, Number of perceived

affordances, Number of emergence events, Number of scaffolding and Number of co-

operative activity.

7.2.1 Thematic Qualitative analysis

Types of interaction – intuitive, non-intuitive and partially intuitive

Intuitive interaction involves utilising knowledge gained through other

experience(s), is fast (Salk, 1983), and generally unconscious (Bastick, 2003). The


coding heuristics employed to code for intuitive interaction are derivations of

methods developed by Blackler et al. (2010). Intuitive interaction does not involve

conscious reasoning (Bastick, 2003) but involves actions and decisions which cannot

be explained or verbalised (Blackler et al., 2010). Children were reasoning

unconsciously when they could not explain how they guided the balls onto the targets

on Osmo or how they arranged different blocks in Monkey Blocks so that they remain

balanced. When children could not verbalise their decisions and actions and were

unable to explain the actual reasons, they were interacting intuitively.

While playing Osmo, one of the participants said:

“It is easy, don’t you understand this?”

One of the children explained the decision to draw lines and curves instead of

using objects to guide the balls onto the targets,

“I like to draw, drawing is easy. Do you know I got an award at the assembly

for art?”

Explaining the strategy to guide orange and green balls onto respective

coloured targets,

“After the green, we will do the orange”.

But the children guided the balls on the targets together, both targets at the

same time.

While playing with Monkey Blocks, one of the children explained the strategy

to stack the blocks in an arrangement as follows,

“…it will keep the blocks balanced.”

“I don’t know.”

Thus, it is evident that there was lack of accurate reportability and verbalisation

in the description of strategies that children used to play the game.

Intuitive interaction is associated with high degrees of certainty, confidence

and expectation with respect to correct use of a feature (Bastick, 2003; Hammond,

1993; Woolhouse and Bayne, 2000). When participants were certain and confident

about their strategy to guide the balls onto the target while playing the game of

Osmo, in contrast to trying out various options, the behaviour was coded as intuitive

interaction. One participant, while playing the game Osmo, said to the other child,


“I know. I know. I got this.”

While playing with Monkey Blocks, children demonstrated certainty and

confidence in their strategic decisions while playing,

“…the stack WILL NOT fall like that..”

The above statements not only show that the children were certain and

confident of their decisions but also were reasoning unconsciously because they did

not verbalise the actual method that they were going to follow.

Non-intuitive interaction is associated with conscious reasoning, uncertainty,

lack of confidence and unclear expectations with respect to the interaction with the

system (Blackler, 2008). Children were reasoning the game strategies consciously,

when they could not figure out the clues in the toys to decide on their next actions to

be performed on the toys.

While playing Osmo, some children could not figure out how to play a level in

the game which asked them to spin fans. They could not interpret the deliberate clue,

a ‘SPIN’ prompt with an arrow pointing towards fan(s).

“Alright, we need to think about this logically”.

While playing with Monkey Blocks, one of the participants while trying to

balance the orange block on its edge (with no weight) expressed concern as the block

kept falling off:

“We need to figure this out slowly….hold this block…I will balance it with

another heavy block…so it does not fall…”

Some children expressed uncertainty, lack of confidence and unclear

expectations while playing with Osmo and Monkey Blocks. While playing Osmo,

some children looking at the ‘SPIN’ prompt, expressed their lack of understanding

and expectations from the level,

“What does this mean?”

One of the participants was unable to determine the boundaries of the game to

place the objects in the physical space to stop the balls from escaping the screen.

“But where does it [ball] go?”


Children when presented with bowls on the screen that function as teleporters

in the game of Osmo, they could not figure out how it works until they saw the balls

falling into one of the bowls, and balls emerging from another bowl.

“What is the point of this level?”

While playing the game of Monkey Blocks, children were faced with situations

where they had to balance the edge of the block. Some children tried to use the blue

(no weight) and the green blocks (weight in the middle), the block kept falling.

“This is hard.”, Patting the forehead indicating confusion.

“How are we going to do this?”

Partially intuitive interactions involved interactions that showed signs of

intuitive as well as non-intuitive interaction. A child while playing Osmo noticed that

the balls were escaping towards the left of the screen instead of being guided towards

the target. He picked up a straw and placed it on the left so that the balls did not

escape anymore. He told the other child,

“Hang on. I have got this! Let’s put this [here] so that balls don’t run away”

The child clearly verbalised his behaviour (non-intuitive interaction) but was

certain and confident about his decision (intuitive interaction). Most children quickly

figured out that the green block is the heaviest and the blue block the lightest in

Monkey Blocks. However, when asked about the orange block, there were mixed

responses,

“It is the same weight as the green block. No, wait a minute; it is lighter

than green block but heavier than blue block. Hmm, I am not sure.”

Children quickly and unconsciously figured out the differences between the

green block and the blue block (intuitive), but demonstrated internal conflict when it

came to the orange block (non-intuitive).


Physical affordances in Osmo were evident when children used spatial

orientation of objects and drawings relative to the balls and targets on the screen to

decide on the optimum angular position to guide the balls onto the targets. Choice of

certain objects for certain alignments was also representative of use of physical


affordances of the virtual elements and the objects. In Monkey Blocks, children used

material and spatial properties of the blocks to stack them in an arrangement.

Children discussing the weights of the blocks in Monkey Blocks,

“This is heavy, I can feel it”

C1: "Oh, this is heavy."

C2 (feeling the blue in her hand): “All are not heavy.”

It is evident from the above that children were using natural clues in the form

of visual (such as spatial orientation) and material properties that children could

perceive to make decisions for their actions.

Perceived affordances is the use of familiarity, past experience and

knowledge about features of physical objects such as mass, elasticity, rigidity,

mobility, etc. and about digital technology. Deliberate clues in the toys helped

children connect them to the prior knowledge and previous experience.

In the game of Osmo, as the objects are moved around in the physical space, it

results in balls being bounced around in the virtual space, hitting targets and objects

mirrored into the virtual space. This is linked to the game of tennis. Such deliberate

clues in the form of actions and movements that link to previous knowledge of

children are Embodied representations.

“…This is like tennis...”

Some children likened the guiding of balls onto the targets to guiding water

through tunnels. These children drew curves in the physical space, referring them as

tunnels, to guide balls onto targets such as spheres, bowls and fans. The embodied

representation of balls hitting the targets and objective of the game to use the

physical space to guide the balls onto the targets helped children to link the context

to the game of tennis.

Referring to the simulation of teleportation, one child explained that he had

read about it in the book, ‘Charlie and the Chocolate Factory’, another one referred

to the ‘Charlie and the Chocolate Factory’ movie. Embodied representations of balls

actually emerging from a bowl when balls were guided to another bowl helped

children connect the level of the game to their experience of watching the movie or

reading the book.


The game Monkey Blocks is incorporated with deliberate clues in the form of

coloured blocks to allow players to decipher the weights in each of the blocks.

Children. However, children used these colour codes to group them while stacking

the blocks but only after they had figured out the difference in weights using natural

clues (physical affordances). The spatial orientation (natural clues) of the stack of

blocks reminded children of buildings. This combined with the knowledge that the

blocks are different in weights, children used their previous knowledge of buildings

such as heavy weights at the bottom and light weights on the top to keep the blocks

in balance.

“….just like buildings…”,

“Heavy bottoms are good…keeps everything in place.”

It is evident from the above that children were using their past experience and

knowledge to relate to things outside the domain of the game, such as stories, tennis,

buildings, tunnels and so on. They were using natural and deliberate clues to make

these connections.

Emergence represents dynamic nature of the system. Children changed their

strategies and techniques in response to contextual clues. In Osmo, children placed

an object in front of the screen along the edge of the tablet screen to block the balls

from escaping from the sides. They erased parts of drawings to guide the balls onto

the target. They started using edges of the sheet of paper to guide the balls onto the

target. Children extended shorter objects by adding another object when they realised

that the object was too short to create the required angular deflection to guide the

balls onto the target. Children were seen partitioning the guns the guns for the two

coloured balls by placing an object between them so that the coloured balls do not

get mixed up. This allowed the children to deal with each of the coloured balls

separately and individually.

Emergence in Monkey Blocks was prominently due to children changing their

strategies and techniques in response to constraints placed on orientation and

placement of the blocks due to weights in them. The emergence is evident from the

fact that children sometimes ended up creating the same arrangement as in the black

and white images (see Figure 21) but with different combination of blocks. Some

children placed another block on the top of a block placed horizontally to balance it

on a single edge.


Scaffolding in using support mechanisms to assist in the game play. In Osmo,

children used hands in V formation to either guide the balls or block the balls from

deflecting away from the targets. While one child was busy aligning the objects, the

other child blocked the escaping from the sides of the tablet screen. Children imitated

strategies of the other child such as use of hands.

In Monkey Blocks, scaffolding was evident when children imitated strategies of

the other child to complete half of a symmetrical arrangement. The children were not

told or prompted about the symmetry, the children automatically detected it even

without discussing between them. Children laid out blocks on the table before

stacking them.

Co-operative activity was evident when children discussed strategies and

worked together physically to play the game. Children were seen strategizing

verbally while playing with Osmo,

“After the green, we will do orange..”

Children worked together to guide the balls onto the targets such as aligning

objects to guide free falling balls onto the targets. One child drawing lines and curves

while the other one erasing it.

While playing with Monkey Blocks, one child held the horizontal block while

another child placed a heavier block on it. Some children completed a part of the

arrangement working in collaboration with the other child.


The coded data with the totalled codes/themes were analysed quantitatively in

SPSS and STATA tools for statistics. The two toys were analysed separately for

intuitive interaction and aspects of Embodiment (i.e. they were not compared with

each other).

Types of interactions

The first part of the analysis investigated types of interactions- intuitive

interaction in the two directly manipulated interfaces—the TEI Osmo, and the

physical toy Monkey Blocks. The boxplots for number of intuitive, non-intuitive, and

partially-intuitive interactions demonstrated by the children with the two interfaces

are shown in Figure 22 and Figure 23.


The TEI game, Osmo, demonstrated the highest number of intuitive

interactions (mean = 30.10, median = 27, standard deviation = 19.74), followed by

non-intuitive interactions (mean = 4.38, median = 3.50, standard deviation = 3.52).

Partially-intuitive interactions were the least demonstrated (mean = 1.62, median = 1,

standard deviation = 1.27).

Monkey Blocks demonstrated a high number of intuitive interactions (mean =

7.88, median = 7, standard deviation = 4.83), while the number of non-intuitive

interactions (mean = 1.57, median = 1, standard deviation = 1.02) and number of

partially-intuitive interactions (mean = 0.62, median = 0.0, standard deviation = 1.10)

were very low.

Figure 22 Number of intuitive, non-intuitive, and partially-intuitive interactions for the TEI Osmo


Figure 23 Intuitive, non-intuitive, and partially-intuitive interactions in the physical toy, Monkey Blocks

A related Samples Friedman test was run to determine if there were any

statistically significant differences between the numbers of intuitive interactions,

non-intuitive interactions, and partially-intuitive interactions in children playing with

each game.

The related Samples Friedman test works by ranking scores of dependent

variables (number of intuitive interactions, number of non-intuitive interactions and

number of partially intuitive interactions), all combined, according to its value, with

the smallest rank assigned to the smallest value. The ranks obtained for each of the

DVs are then averaged. This results in a mean rank for each of the DVs. The null

hypothesis of the Friedman test is that the mean rank is the same for all the DVs.

However, if one of the DVs tends to have higher values than the other DVs, it will

have been assigned higher ranks and will have a higher mean rank (and vice versa for

the DV with lower scores). It is this difference in mean rank that is tested by the

Friedman test for statistical significance. This approach was used to determine

whether the differences in the DVs are statistically significant.

The mean ranks obtained for each of the dependent variables, numbers of

intuitive, non-intuitive and partially-intuitive interactions are presented in Table 9.


Table 9 Mean Rank Values of Number of Intuitive Interactions, Non-Intuitive Interactions and

Partially Intuitive Interactions

The mean rank values of intuitive, non-intuitive, and partially-intuitive

interactions were not same for both Osmo and Monkey Blocks. Thus, the differences

number of intuitive interactions, number of non-intuitive interactions and number of

partially intuitive interactions are statistically significant for Osmo, χ2(2)= 54.544, p

< 0.05 and Monkey Blocks, χ2(2)= 75, p <0.05.It is therefore evident from the results

above that both Osmo and Monkey Blocks demonstrate high numbers of intuitive

interactions.

Primary predictors of intuitive interaction

Having established that both Osmo and Monkey Blocks are substantially

intuitive, the second part of the analysis investigated primary predictors (in terms of

aspects of Embodiment) of intuitive use. Multiple regression analysis was used to

determine the proportion of variation in intuitive interaction that could be explained

by aspects of Embodiment.

A multiple regression was run to explain how much of the variation in intuitive

interaction with each toy could be explained by aspects of Embodiment. All predictor

variables (physical affordances, perceived affordances, emergence, scaffolding, and

co-operative activity) were added to the regression models; in other words, no

variables were dropped from the models. The partial regression coefficients and

Variation Inflation Factor (VIF) values are given in Table 10. The VIF values

obtained in regression analysis carried out for Osmo and Monkey Blocks were mostly

between 1 and 5, indicating very little multicollinearity between the predictors.

Physical affordances exhibited moderate multicollinearity, with a VIF value between

5 and 10.

Type of interactions

Mean Rank Values Osmo Monkey Blocks

Number of Intuitive Interactions

2.81 2.95

Number of non-intuitive interactions

1.94 1.52

Number of partially intuitive interactions

1.25 1.52


Table 10 Regression coefficients and VIF values for the MRS system

The aspects of Embodiment statistically significantly explained 52.4% of the

variability in intuitive interaction with the Osmo game, F(5,36) = 27.28, p<0.05; and

51.5% of the variability in intuitive interaction with Monkey Blocks, F(5,36) = 26.04,

p<0.05. Comparing the relative contributions of the aspects of Embodiment to the

intuitive interaction with the two systems, it was found that physical affordances

explained 36.06% of the variability in the intuitive interaction with the Osmo game.

The results for the contribution of all aspects of Embodiment are shown in Figure 24.

Similarly, in the case of Monkey Blocks, physical affordances explained 42.42% of

the variability in intuitive interaction. The contribution of all aspects of Embodiment

is shown in Figure 25.

Toy Predictor variables -aspects of Embodiment

Partial Regression Coefficients

Variation Inflation Factor (VIF)

Osmo Physical affordances 0.3603 8.925 Perceived affordances 0.3918 1.527 Emergence 0.2549 3.670 Scaffolding 0.1602 4.948 Co-operative activity 0.6918 3.418

Monkey Blocks

Physical affordances 0.4242 6.614 Perceived affordances 0.1347 1.808 Emergence 0.2155 3.750 Scaffolding 0.1232 2.853 Co-operative activity 0.099 5.585


Figure 24 Relative contributions of aspects of Embodiment to intuitive interaction with TEI game

Figure 25 Relative contributions of aspects of Embodiment to intuitive interaction with Monkey Blocks

Due to multicollinearity, the contributions of aspects of Embodiment to

intuitive interaction do not total 100%. The existence of multicollinearity and the


VIF values indicate that a certain level of correlation exists between the predictors,

especially between physical affordances and other predictors. A Spearman's rank-

order correlation was run to assess the relationship between the predictors (Table 11).

Table 11 Correlations between aspects of Embodiment (predictors)

For Osmo, there was a strong positive correlation between physical affordances

and emergence, rs(40) = 0.867, p < .01; physical affordances and scaffolding, rs(40) =

0.952, p < .01; and physical affordances and cooperative activity, rs(40) = 0.728, p <

.01. There was a moderate positive correlation between physical affordances and

perceived affordances, rs(40) = 0.506, p < .05.

For Monkey Blocks, there was a strong positive correlation between physical

affordances and emergence, rs(40) = 0.755, p < 0.01; physical affordances and

scaffolding, rs(40) = 0.801, p < 0.01; and physical affordances and cooperative

activity, rs(40) = 0.885, p < 0.01. There was a weak positive correlation between

physical affordances and perceived affordances, rs(40) = 0.375, p < 0.01.

There was a weak positive correlation between perceived affordances and other

aspects of Embodiment. There was a strong correlation between emergence and

Toy Predictors of intuitive interaction

Physical affordances

Perceived affordances

Emergence Scaffolding Co-operative activity

Osmo Physical affordances

0.506 0.867 0.952 0.728


0.506 0.388 0.461 0.403

Emergence 0.867 0.388 0.825 0.564

Scaffolding 0.952 0.461 0.825 0.731

Co-operative activity

0.728 0.403 0.564 0.731

Monkey Blocks

Physical affordances

0.375 0.755 0.801 0.885


0.375 0.665 0.310 0.293

Emergence 0.755 0.665 0.777 0.715

Scaffolding 0.801 0.321 0.777 0.794

Co-operative activity

0.885 0.293 0.715 0.794


scaffolding, emergence and co-operative activity, and scaffolding and co-operative

activity.

7.3 DISCUSSION

The TEI, Osmo, consisted of both physical and virtual elements, while the

physical toy, Monkey Blocks, consisted of physical elements only. The results in

Section 7.2.2 indicate that both games demonstrated more intuitive interactions than

non-intuitive and partially-intuitive interactions.

Results suggest that physical affordances were the primary predictor of

intuitive interaction in both games. Perceived affordances were the second highest

predictor of intuitive interaction with the Osmo game, but was the lowest contributor

to variability in intuitive interaction with Monkey Blocks. Emergence, scaffolding,

and co-operative activity were the next highest contributors to variability in

children’s intuitive interaction when playing with both games, and showed high

correlation with physical affordances.

Children used physical, material, and spatial properties of physical and virtual

elements in the systems to interact intuitively with them. They used the spatial

orientation of the targets and the balls on the screen (virtual elements), and the

physical and material properties of the objects in front of the screen, to obtain the

optimum angular position of the objects to guide the balls onto the targets. With

Monkey Blocks, the children used the spatial and material properties of the blocks to

stack them in an arrangement.

The materiality of the blocks played a crucial role in children’s play with

Monkey Blocks. They figured out that the orange and green blocks were heavier than

the blue blocks by feeling their weights. Most children, however, could not figure out

the difference between the orange and green blocks. While the two blocks weighed

the same, a weight was embedded at one end of the orange block, and another at the

centre of the green block. The children’s perception of the weights of the blocks,

therefore, depended on the way they held them in their hands. They were observed

holding the blocks in their two hands, and then holding their ends to compare the

weights of the blocks. Some children figured out the heavier end of the orange block;

some believed that the orange block was heavier than the green block; some believed

that the green block was heavier than the orange; and some believed that there was


no difference between the two. None of the children said that the two blocks weighed

the same. Children developed their strategies to complete the arrangement according

to their own interpretation of the weights.

Perceived affordances were the next prime contributor to variability in

children’s intuitive interaction with the Osmo game. In the absence of real

affordances in the virtual elements, they resorted to interpreting deliberate clues in

the system. They used their past experience and knowledge to interpret the deliberate

clues in the virtual elements. For example, some children had personally experienced

similar scenarios of matching objects to same-coloured targets while playing other

games.

Most of the deliberate clues in the Osmo game were simulations of

functionality of elements in action that were known to children from their past

experience and knowledge. Simulations of the functionality of a teleporter, spinning

fans, and targets disappearing when hit by the balls, are examples of simulated

actions. The perceived affordances was based on the experience and knowledge of

how things (such as teleporters) work. Children were able to identify the

functionality of the teleporter when they noticed that the balls guided to a bowl

resulted in balls emerging from another bowl. Some children had never heard of

teleporters; however, they were still able to detect the functionality of the deliberate

clues simply by seeing it in action. Pezzulo (2011) referred to such deliberate clues

as ‘Embodied representations’. Children perceived these deliberate clues by using

their past experience and knowledge.

The simulations of the virtual elements were often initiated through physical,

direct manipulation of objects. Children were using spatial orientation of the virtual

elements—such as balls and targets on the screen and the spatial and physical

properties of the objects—to decide on the manipulations of the objects. This meant

that while the manipulations of the physical objects were determined by the physical

and spatial properties of the objects, they also required interpretation of the deliberate

clues in the virtual elements. Gaver (1991) referred to such affordances as ‘sequential

in time affordances’. This explains the moderate correlation between physical

affordances and perceived affordances in the Osmo game.

Perceived affordances was the lowest contributor to the variability in intuitive

interaction with the Monkey Blocks. Children mostly relied on the natural clues in the


form of physical and material properties of the blocks to create an arrangement.

Although some children could not detect the difference in weights and location of

weights in the blocks, they were still able to form an arrangement, using the physical

affordances of the blocks. Those who were able to detect the difference in weights in

the blocks, and the location of the weights inside the blocks, were using their past

experience playing with blocks and their knowledge of weights. Children arranged

lighter blocks on the top of the heavy blocks. When a block had to be balanced

horizontally on one end, some children placed a heavier block on top of the end of

the horizontal block. One child compared this to the functionality of a paper weight.

Some children said that they learned about heavy and light weights at school,

but they could not explain exactly what and when. Studying the attributes of heavy

and light objects is covered in the early numeracy and mathematical understanding

section of the Queensland education curriculum (Queensland Studies Authority,

2006). Some children used the orange block as the horizontal block, and balanced it

by placing its heavier end on the vertical block. The knowledge required to interact

intuitively with Monkey Blocks, a system with only physical elements, was based on

facts pertaining to weights that they had learnt at school. These facts were not

revealed through physical manipulation of the blocks at the start of the game at the

least. This explains the weak correlation between physical affordances and perceived

affordances in the Monkey Block game.

There was a strong correlation between physical affordances and other aspects

of Embodiment, namely, emergence, scaffolding, and co-operative activity for both

the games. The environment is the crucial influencing factor in developing this

relationship, as children use elements in the environment to play together. They

manipulated objects to effect change in the layout and structure of the elements in the

physical and virtual spaces (physical space only in the case of Monkey Blocks). This

resulted in change in affordances offered by elements in both spaces. This, in turn,

changed the manipulations of the objects. Some of the changes in affordances

resulted in manipulations of the objects that were not anticipated by the game

designers. Thus, emergence was the next most important contributor to variability in

children’s intuitive interaction.

Scaffolding and co-operative activity were the next most important

contributors to variability in intuitive interaction. Children used their hands, and the


space and people around them to support their play. They used symmetry of elements

in the game play to imitate another child’s strategies and play. The imitation was a

confident decision, rather than a vague mimic of another child’s actions. Children

played together, co-operating with each other, or getting out of the way if they were

impeding another child’s play. They developed strategies focussed on reaching a set

goal together. There was a high positive correlation between physical affordances

and scaffolding, and between physical affordances and co-operative activity. The

physical and material properties of elements in the system allowed children to

support each other’s game play, and to co-operate and collaborate.

In conclusion, physical affordances was shown to be the prominent facilitator

of children’s intuitive interaction with TEIs and physical products. This, in turn, has

an impact on emergence, scaffolding, and co-operative activity due to the correlation

of these aspects with physical affordances. These aspects are important because they

could have an impact on people’s intuitive experiences; this topic, however, remains

an area for future research. Products could be designed for intuitive interactions but

that depends on the how the deliberate clues are used in design. This statement is

supported by the fact that Embodied representations in deliberate clues have emerged

as a prominent factor in the use of perceived affordances in intuitive interactions with

TEIs with virtual elements. One of the ways of incorporating Embodied

representations is through the use of sequential affordances.

7.3.1 Implications for Design

Based on the above discussion, the following guidelines were developed to

assist in designing Embodied intuitive products for children:

Physical affordances should be used in design to offer natural clues for intuitive

interaction - Natural clues allow children to use their sensorimotor knowledge and

Embodied experiences for intuitive interaction. This study found that intuitive

interaction in both physical products and TEIs is primarily due to physical

affordances. It is the direct interaction and manipulation of the physical elements that

facilitates intuitive interaction. In physical products, the feedback from the system is

in high conformance with the action performed on the system. Thus, TEIs, where

physical and virtual elements co-exist, could be designed for intuitive interaction by

facilitating interaction with the system through interactions with the physical

elements that allow children to take advantage of their affordances. The virtual


system in the TEI (if present) could be used to provide feedback on the manipulation

of, and interaction with the physical elements.

Where physical affordances cannot be used in design, perceived

affordances should be used to offer deliberate clues for intuitive

interaction – Clues, both deliberate and natural clues allow children to use

their experiential knowledge for intuitive interaction. To facilitate this

interaction, virtual elements in the interface should be designed with

children’s past experience and prior knowledge in mind. However, this is

easier said than done, as children accumulate different experiences and

knowledge. To allow children to interpret the symbolic clues correctly (to

avoid any false affordances), scaffolding could be provided. For example,

in Osmo, children found it difficult to determine how to spin a virtual fan,

even though symbolic clues in the form of a ‘SPIN’ prompt were provided

on the virtual interface. Children were only able to figure out how to spin

the fan when the balls started to fall on it, and it began to spin. Embodied

representations such as animated clues (Uden & Dix, 2000), that is, clues

that depict or represent the action to be performed, are preferred over

symbolic clues, as children can use both their sensorimotor and

experiential knowledge to interpret and understand these deliberate clues.

Interactions in physical and virtual spaces in TEIs should be in the

same dimensions - This guideline mostly applies to TEIs, where the

coupling of the physical and virtual poses the biggest challenge (physical

products do not suffer from coupling issues). One of the ways to resolve

coupling issues is to segregate the physical and virtual spaces so that

interactions in both spaces are in the same dimensions. For example, if the

interactions in the virtual space are in two dimensions (such as on a tablet

screen), limiting the interactions and manipulations in the physical space

to two dimensions (such as by moving objects on a horizontal plane such

as a table) helps children to traverse the coupling between the physical and

virtual spaces.

Use both physical and perceived affordances (sequential in time,

or/and nested in space) to couple physical and virtual spaces in TEIs -

Again, this guideline mostly applies to TEIs. Actions performed in the


physical space (in response to physical affordances of the physical

elements) result in clues for actions in the virtual space. These are

‘sequential in time’ affordances. Affordances could also be nested in

space; that is, where both physical and perceived affordances are present at

the same time but in different locations (one in physical space and the

other in virtual space, for example), and are linked to the same objective or

action that they afford.

The size of physical and virtual interaction spaces should be calibrated

for boundary limits - Separating the virtual and physical space in a TEI

has its own limitations. The size of each space in TEIs is usually not the

same. This could result in physical manipulations and interactions in the

physical space going beyond the limits of the virtual space. One way to

resolve this is to assign boundaries to the physical space in relation to the

virtual space. For example, in Osmo, a simple calibration of the physical

space with respect to the virtual space, and drawing lines on the table to

limit manipulations within these boundaries, would solve the problem.

Alternatively, a feedback from the system to move back within the

boundary limits could help train people to determine the boundaries.

7.4 SUMMARY

Having established (in Chapter 6) that physical products have aspects of

Embodiment which allow children to intuitively interact with them, this chapter has

presented a within-subject study of two systems: one physical and the other a TEI,

both of them requiring interaction with physical objects to achieve game objectives

and goals. Experiment 2 determined which of the design aspects of Embodiment

discussed in Section 4.2, have a significant impact on children’s intuitive

interactions. The TEI game Osmo had virtual elements as well as the physical

elements, while the physical game, Monkey Blocks consisted of physical elements

only.

The results show that physical affordances was the primary predictor of

intuitive interaction with both systems. Perceived affordances was the next most

significant predictor of intuitive interaction with the Osmo game, but contributed

least to variability in intuitive interaction with Monkey Blocks. The prominent use of


perceived affordances in Osmo was due to the use of Embodied representations

incorporated in the design through sequential affordances. On the other hand,

intuitive interaction with Monkey Blocks was mostly due to natural clues in the

system. Children used their past experience and knowledge to play with both Osmo

and Monkey Blocks.

Emergence, scaffolding, and co-operative activity were the next most

significant contributors to variability in children’s intuitive interaction with both the

systems, and showed a high correlation with physical affordances. These

relationships establish the scope for future research in the use of these aspects to

facilitate intuitive experiences.

Chapter 8 now discusses the design implications of the results of Experiment 1

and Experiment 2. The findings from Experiment 1 and Experiment 2 were

integrated into Blackler's (2008) continuum of intuitive interaction. The Model of

Embodied Intuitive Interaction (MEII), an interaction model that provides a reference

for the design and development of Embodied intuitive products for children is also

discussed.


Chapter 8: Discussion

This research study has investigated children’s Embodied intuitive interaction.

Physical products, virtual interface, and a TEI were used to study children’s

Embodiment and intuitive interaction. Experiment 1 (Chapter 6) compared a physical

Jenga and a virtual interface for Embodiment and intuitive interaction. The results

suggest that the children interacted more intuitively with the physical Jenga than the

virtual interface. In terms of Embodiment in children’s interaction with the two

products, physical affordances, due to the spatial and material properties of the

physical artefact, contributed to intuitive interaction with the physical Jenga. The

intuitive interaction with the virtual interface, on the other hand, was mostly due to

perceived affordances.

Experiment 2 ( Chapter 7) investigated aspects of Embodiment that contribute

to variability in children’s intuitive interaction with physical product and a TEI. The

findings suggest that physical affordances was the primary contributor to intuitive

interaction with both the products. Perceived affordances was the second most

contributor in Osmo while it was the least contributor in Monkey Blocks. Embodied

aspects could thus facilitate intuitive interaction and efforts should be made to

incorporate these aspects in design.

This chapter discusses the design implications of the results of the two

experiments (in Section 8.2). The results are discussed in relation to the continuum of

intuitive interaction, and the findings are integrated into this continuum. Section 8.2

discusses a model for children’s Embodied intuitive interaction—MEII. This model

provides a reference for designers and researchers to develop Embodied intuitive

products for children.

8.1 DESIGN IMPLICATIONS

This study has found that physical products have properties that make them

intuitive to use (Desai et al., 2015). It further found that there are more aspects of

Embodiment used in children’s intuitive interaction with physical products than with

virtual interfaces. Physical affordances is the primary contributor to children’s

intuitive interaction with physical products, while they used perceived affordances to


intuitively interact with the virtual interface. Other aspects of Embodiment—

emergence, scaffolding, and co-operative activity—showed correlations with

physical affordances. These results can be explained in relation to the continuum of

intuitive interaction (Section 8.1.2).

8.1.1 Children’s intuitive interactions with physical and virtual interfaces

Physical products are associated with low domain transfer distance, the

distance between the application domain and the origin of prior knowledge

(Diefenbach & Ullrich, 2015). In physical products, the origin of children’s prior

knowledge, and the application of that knowledge, both relate to the same physical

domain with spatial and material characteristics. Low transfer distance results in less

verbalisation, and effortless use of the interface (Diefenbach & Ullrich, 2015). Less

verbalisation (Swaak & De Jong, 2001) and effortless use (Alter, Oppenheimer,

Epley, & Eyre, 2007), in turn, are associated with intuitive thinking. Thus, low

transfer distance in physical products results in intuitive interactions in children.

Maier and Fadel (2009) suggested that the relationship between artefacts

contributes to affordances, as the interaction between the artefacts offers various

possibilities for action. Maier et al. (2009) referred to this relationship as ‘Artefact-

Artefact-Affordances (AAA)’. Although Maier et al. described AAA as a

relationship between artefacts that are identical (for example, a chair stacks on

another chair), the concept can be extended to artefacts that are not identical, such as

the mix of physical objects and virtual elements in Osmo. Physical Jenga blocks

consist of identical blocks. The spatial and material properties of the blocks and the

stack provided the children with various possibilities for action, as did the other

blocks. The spatial layout of the stack offers possibilities for other blocks to be

placed in certain positions ( Figure 26). In Monkey Blocks, the blocks are colour-

coded as per the location of weights inside them, and the children feel the material

difference. In this case, the material and spatial properties of the blocks determine the

possible actions. Children feel the weights of the blocks, and place them horizontally

on the vertical stack. Then they slide the blocks at different angles until they feel that

the block will remain stable (Figure 26, top).


Figure 26 Artefact-Artefact Affordances: stack-ability and slide-ability of blocks in Monkey Blocks (above); pull-ability and slide-ability of blocks in Jenga (below)

There are three types of artefact-artefact interactions in TEIs - physical to

physical, physical to virtual, and virtual to virtual. In Osmo (Figure 17), the geometry

of physical objects (such as pencils and erasers) allows other similar objects to be

aligned with them to deflect the virtual balls. A copy of the physical manipulations is

created in the virtual space, and facilitates physical-virtual interaction in the system.

The physical manipulation of objects (physical-to-physical interaction) then results in

changes to the virtual elements (virtual-to-virtual interactions). The virtual balls, for

example, are deflected to hit virtual targets.

Children’s actions on physical objects, and the objects’ responses, are spatially

compatible; therefore, the manipulation of these physical objects and their responses

are easily anticipated. This results in high degrees of conformance (Beaudouin-

Lafon, 2004)—the extent of similarity between a user action on a domain object and

the resulting response. Sliding a block in the physical Jenga toy to the left, moves the

block to the left. The response to children’s action in virtual interfaces depends on

the design of the interface, the technology (e.g. touchscreen), and the system

configuration. Children are thus less able to reliably anticipate the response from a

virtual system. The high degree of conformance in physical products, on the other


hand, results in anticipated responses to manipulations of the blocks; this, in turn,

results in a higher number of intuitive interactions.

Virtual interfaces are instrumental interactive systems, where a computing

element is treated as a tool that acts as a mediator between users and virtual elements

(Beaudouin-Lafon, 2004). In the virtual app, children’s interaction with the stack of

blocks is through the touch screen interface of the tablet. The tablet acts as a

mediator between the children and the virtual stack of blocks. Children are

effectively interacting with two systems simultaneously—the tablet, which has its

own system of operation and perceived affordances (e.g. swiping) and the game,

which has perceived affordances transferred from other games and elsewhere (e.g.

colour codes based on population stereotypes). This is likely to result in high degrees

of indirection, and a high transfer distance. Physical products, on the other hand, do

not require any mediators; rather, they involve direct interaction with the system and

its elements.

The actions offered by the mediator, and the actions offered by the virtual

elements, represent a relationship between the artefact and the children. This is

referred to as Artefact-Children-Affordances (ACA) (Maier & Fadel, 2009). Often,

these actions are contradictory, resulting in unsuccessful or incorrect interactions.

For example, tapping on the blocks in the Jenga app, and swiping left and right on

the tablet screen, were contradictory actions resulting in the stack falling. This

suggests that the elements of a system should either invoke consistent actions, or the

app should be designed to take this inconsistency into account. The blocks and the

stack in the virtual Jenga game offer the possibilty of actions such as push, pull, and

stack on the other blocks (AAA); however, children were found swiping their fingers

on the tablet, and tapping on the blocks (ACA). It was only the tap action in the app

(ACA) that was consistent with the push action with the physical Jenga blocks

(AAA). This means that the actions that elements offer to other elements in the

system (AAA) should be consistent with the actions that they offer to the children

(ACA).

8.1.2 Relationship with the continuum of intuitive interaction

Building upon past work (Blackler & Hurtienne, 2007) that compared and

contrasted the two separate continua of intuitive interaction (as discussed in Section

2.3.2), the results from Experiment 1 and Experiment 2 have been used to present an


Enhanced Framework for Intuitive Interaction (EFII) (Blackler et al., 2018). Figure

27 is adapted from EFII, where children’s Embodied intuitive interaction with

physical products, virtual interfaces, and TEIs are illustrated in relation to Blackler's

(2008) continuum of intuitive interaction and the German-based Intuitive Use of

User Interfaces (IUUI) research group’s continuum of knowledge in intuitive

interaction. The aspects of Embodiment discussed in this research study are

highlighted in red in Figure 27, which presents their relationship with the two

continua. The latter are both shown in grey. The Enhanced Framework for Intuitive

Interaction (EFII) highlights parallels and connections between the different

dimensions of intuitive interaction shown on the right-hand side - pathways to

intuitive use, interface types, characteristics of features, and the origin of the

knowledge that enables intuitive use.

Figure 27 Research results incorporated into an Enhanced Framework of Intuitive Interaction (EFII) (adapted from Blackler et al., 2018)

Blackler’s (2008) continuum of intuitive interaction (Figure 2) is re-

conceptualised as ‘pathways to intuitive use’ and shown alongside is the ‘origin of

knowledge that enables intuitive use’ (Figure 27). The level of physicality and level

of virtuality determines the pathways to intuitive use, characteristics of features and

type of knowledge leveraged. Physical products are on the extreme left of EFII,


while virtual interfaces are on the extreme right. TEIs are between these extreme

ends, and represent products with both physical and virtual elements. TEIs towards

the left extreme represent TEIs with more interactions with the physical, while TEIs

towards the right comprise more interactions with the virtual.

Children mostly relied on physical affordances to interact with the products

that were more physical (such as grasping, holding, taping, and sliding the physical

blocks). They mapped the physical and material properties of the objects (such as

looseness, shape, weight) onto decisions and actions (for example, to remove, stack,

and align the objects). Experiment 1 found some use of perceived affordances with

the physical toy; however, most of the overall uses and intuitive interactions were

facilitated by physical affordances. Children can, however, also leverage cultural

conventions and metaphors (e.g. turning a wheel, using a racket to play tennis,

balancing blocks one above the other to create a stack).

In contrast, children mostly used perceived affordances to interact with

interfaces towards the right of the EFII which, depending on the system and its

design, are more virtual. Virtual elements within these interfaces do not have real

physical affordances. Children thus rely on their past experience and knowledge

acquired from playing other games, both physical and virtual. Children were not only

interacting with the virtual elements within the virtual interface, but also with the

mediating element (e.g. the tablet in Experiment 1). They used cultural conventions

associated with tablets (such as swiping left-right on the screen), and conventions

associated with physical blocks and stacks (such as tapping on the blocks).

Children’s intuitive interaction with virtual interfaces is thus associated with

population stereotypes (e.g. the colour codes in the app) and perceived affordances

on the EFII (Figure 27).

Physical products (left of the EFII in Figure 27) are associated with low

domain transfer distance and high degree of conformance. As interfaces become

more virtual, transfer distance could increase and the degree of conformance could

decrease, depending on the way physical and virtual elements are configured within

the system. Familiar features and metaphors are used as the pathways to intuitive

interaction in interfaces as they become more virtual. Presence of natural clues

become less as interfaces become more virtual and are replaced with deliberate clues

which rely on familiarity and previous knowledge of the users to determine the


actions afforded by these clues. This is generally the case with more complex

systems that cannot be simply mapped at the sensorimotor level (such as accounting

software or menu-driven role playing games). In these cases, the level of previous

domain experience is more relevant in determining the intuitive potential of the

system. Again, however, metaphors could be used as a bridge to more ubiquitous

knowledge.

In relation to directly manipulated TEIs (shown in Figure 27), children

primarily used physical affordances in intuitive interaction with the physical

elements in TEIs, and perceived affordances to interact with the virtual elements of

the system. Thus, they could access any part of the continuum to interact intuitively

with TEIs, depending on the design and configuration of the physical and virtual

elements in the system. Emergence, scaffolding, and co-operative activity are

represented on the extreme left of the EFII, as Experiment 2 showed that they are

strongly correlated to physical affordances. The EFII thus suggests that children’s

Embodied intuitive interaction is mostly related to the left of the framework, and is

mostly associated with sensorimotor knowledge.

Children use their Embodied experiences and sensorimotor knowledge to

intuitively interact with physical products towards the left of the EFII. This

experience and knowledge is available everywhere, and at any time, to assist them to

interpret the natural clues to possible action on the physical elements that physical

affordances offer them. This explains the high ubiquity of previous experience for

the left end of the EFII, for more physical products. Population stereotypes are the

next most ubiquitous in terms of availability of experience and knowledge to assist

children to interact intuitively with the interfaces. Cultural experiences and

knowledge within a group of children are readily accessible and available. Ubiquity

of previous experience decreases towards the right end of the EFII, as children might

or might not recognise the familiar features, depending on their pattern of previous

experience. Children might not have the knowledge to interpret the deliberate clues

that perceived affordances must offer in order to interact intuitively with the

interface. Therefore, ubiquity is important for children as they have less overall

experience to draw on.

Metaphor is an exception here, as image schemas and Embodied metaphors

(for example ‘up’ to increase, ‘progress’ to travel along a path) could be more


ubiquitous as they could be derived from children’s sensorimotor knowledge and

Embodied experiences. Hurtienne, Klöckner, et al. (2015) showed that image

schemas can be more ubiquitous because they use metaphorical extensions of basic

mental concepts and, hence, can be intuitive as well as inclusive. Image schemas are

arguably universal, so sit on the far left. Metaphor has been slightly detached from

the other parts of Blackler & Hurtienne's (2007) original continuum (Figure 2). This

is because it is not always a simple continuation of the other concepts and, in fact,

could be applied in other ways than originally assumed by Blackler & Hurtienne

(2007). The extension of the metaphor block beneath the continuum is intended to

demonstrate that metaphor can, in fact, be applied through both physical and

perceived affordances.

Children could easily interpret some metaphors if they are derived from

everyday experiences, while other metaphors could require them to develop expertise

and understanding to be able to use them intuitively. For example, the verbal and

symbolic reference to all that is good as ‘up’ or ‘high’, such as describing good

evaluations as ‘high marks’, or a ‘thumbs up’ gesture (Meier & Robinson, 2004); and

the use of a letter box icon for email on an interface developed for children (Uden &

Dix, 2000). However, metaphors such as a cracked wine glass to convey fragility

require children to develop an understanding through practice and learning. Uden &

Dix (2000) found that children were unable to interpret icons with images of objects

that they have not encountered before (such as a typewriter), or objects that they do

not associate with the activity (such as a fountain pen). Such metaphors could be less

ubiquitous in experience and knowledge.

8.2 MODEL FOR EMBODIED INTUITIVE INTERACTION (MEII) FOR CHILDREN

One of the outcomes of this research study is the model for Embodied Intuitive

Interaction (MEII), an interaction model to guide the design of children’s products

that facilitate Embodied intuitive interactions. Buur & Andreasen (1989) described

design models as representations of the properties of products that designers create.

These representations provide insights into the properties of the product that is being

designed (such as insights into products’ usability, sustainability, ergonomics and so

on). Interaction models represent ways in which interactions with certain

properties—such as intuitiveness (Blackler, 2008); Embodiment; thoughtfulness


(Löwgren & Stolterman, 2004); seductiveness resulting in playful, fun, and effective

user experiences (Anderson, 2011)—can be designed. Interaction models take

various forms such as mathematical formulae, verbal descriptions, sketches, block

diagrams, and functional models, depending on the expertise of the designer and the

abstraction level of the design process. The model for children’s Embodied intuitive

interaction is presented in the form of two block diagrams, one for perception (Figure

29), and the second for action (Figure 30). The derivation of these models is

discussed below.

A systems-based approach is used to model children’s Embodied intuitive

interactions. This approach allows the representation of relationships between

systems and sub-systems, as well as representation of children’s activities on

individual sub-systems or whole systems (Churchman, 1971). If a sub-system is

further investigated in future research, it can be represented by more subsystems. The

model would then allow encoding of the qualities and properties of the interactions

into a formal system (mathematical objects), design prototypes, or design tool kits. It

would then be possible to verify system's properties using reliable methods such as

formal methods (for mathematical encodings) and usability methods.

Section 8.2.1 discusses the development of the model and Section 8.2.2

describes the perception (Figure 29) and action models (Figure 30).

8.2.1 Embodied Cognition as a perceptual system

Embodiment theory claims that thoughts and actions are the results of the brain

and the body working together in cognitive information processing. This has been

discussed in the literature review (Section 2.1). Wilson & Golonka (2013) described

Embodied cognition as a process of a continuous loop of perception and action in an

environment. The perceiver deciphers the environment and makes decision on the

actions to be performed on the environment. The brain plays an important role as

well, but the environment is the originating source of the perception action loop and

not the brain (Wilson & Golonka, 2013). The brain and the body (together

represented by the perceiver) and the environment, all work together as a dynamical,

perceptual system (Hinton, 2014) which is presented in Figure 28.

Children are compulsive interpreters; they try to sense and perceive everything

around them (Andersen, 2001). Rosen (2012) describes this tendency as


characteristic of a perceptual system. Children interacting with the environment,

perceiving it and acting upon it, form a basic cognitive unit (Gaines, 1989), as shown

in Figure 28. Perceptual systems are internal predictive models of themselves and/or

of their environment that utilise the models’ predictions to control their present

behaviour. Children and all biological systems assess their current change of state

depending not only on past and present circumstances, but also on the future. Such

systems are referred to as ‘natural perceptual systems’.

Children perceive the qualities in the external world through their sensory

systems. Changes to sensory systems due to change in the external environment

result in percepts; that is, mental impressions of things perceived by the senses

(Rosen, 2012). Children process and organise the relationships among these percepts,

and then interact with the external environment through effector mechanisms (such

as hands). Such interactions cause changes in the external environment, which are

then re-perceived (Figure 28).

One of the qualities of perceptual systems is that they are socially distributed as

they experience the environment through physical objects and social activities with

other organisms in the world. Perceptions and actions with the social part of the

environment are evaluated differently to those in the physical artefact part of the

environment (Gaines, 2013). The primary objective of any social interaction is the

explanation and understanding of human actions. Children as distributed perceptual

systems, and the separation of experiences in the environment into physical and

social experiences, are shown in Figure 28.


Figure 28 Embodied Cognition as distributed perceptual systems, (adapted from Gaines (1989) and Hinton (2014))

The social experiences in the environment represent the other child with whom

every participant was paired in Experiment 1 and Experiment 2. Children-to-social

world interactions in this research study are children-to-children interactions that

occur through the cooperative activity aspect of Embodiment (that is explained in

Section 8.2.2). Perceive and act form the basis of the sensory perceptual systems that

process children’s sensorimotor knowledge, Embodied experiences, and experiential

knowledge. Thus, distributed perceptual systems are ideal starting blocks for

representing the findings of this research study in an interaction model. The MEII

consists of two parts: one representing children perceiving (Figure 29) and the

second one representing acting on artefacts and the social world (that is, other

children and the environment) (Figure 30). These two parts of the model are

discussed below.

8.2.2 Incorporating Embodied aspects of children’s intuitive interaction into the model

Direct interaction and manipulation is associated with a direct relationship

between a child’s action on an object and the outcome of that action. The Model for

Embodied Intuitive Interaction (MEII) represents children perceiving (Figure 29) and

acting (Figure 30) on artefacts and the social world (that is, other children and the

environment). The perceive side of the MEII (i.e. children’s perception of the

physical world and the social world) is shown in Figure 29. Artefacts represent


products such as physical products, TEIs, and virtual interfaces (shown on physical

virtual continuum of Figure 1).

Figure 29 Perception side of the Model for Embodied intuitive interaction (MEII) in children

Affordances, both physical and perceived, offer children clues as to how they

might interact with products. Children used natural clues that represent the natural

properties of the elements within the artefacts to interpret actions physical

affordances. On the other hand, children used deliberate clues that are incorporated

into the design of artefacts and natural clues to interpret perceived affordances. This

enabled children to understand, learn, and interpret the interactions that they need to

perform on the artefacts. Two types of deliberate clues have been identified in this

research study: Embodied representations that are simulations of the functionality of

symbols (for example, the simulation of balls hitting a fan and, in turn, resulting in a

spin of the fan in the game of Osmo); and symbolic clues (for example, colour codes

on the blocks and the crash symbol to warn of the danger of the stack falling over in

the Jenga app) that depend on children’s past experiences with such symbolic

representations.


Both natural and deliberate clues are important. If these clues are not reliably

and accurately interpreted for an appropriate interaction, the affordances could either

remain hidden, or result in actions that are incorrect or/and unexpected (false

affordances). For example, swiping left and right on the screen in the Jenga app

rotates the stack of blocks 360 degrees; however, the affordances remained hidden as

children could not interpret and understand the clues to swipe for the intended

purpose. While playing Osmo, some children tapped on the balls on the tablet screen

to direct them to the targets, instead of using the physical objects in the physcial

space. This was a false affordances that children quickly learned about. They then

started playing the correct way, manipulating the physical objects in the physical

space. Hidden and false affordances can result in mistakes and confusion, and thus

contribute to non-intuitive interactions.

Children use epistemic actions on artefacts to find solutions more quickly and

easily. This process frees their minds of the cognitive load. For example, children

moved the physical objects around in Osmo to find the right alignment and

distribution of objects to guide the balls onto the target in the virtual space. Similarly,

they were also seen touching and feeling the Jenga physical blocks to determine

which block to remove from the stack. Thus, scaffolding is represented in the

perception section of the MEII (as shown in Figure 29).

Children work with each other to perceive and decide on the actions to be

performed on the artefacts. Thus, the relationship between children and the social

world during perception phase is through co-operative activity (Figure 29). Children

work with other children and the environment to interact with the artefact. When

children are involved in social play with another child, their interactions with the

game elements are influenced by the interactions of the other child. Each of their

interactions leaves a clue (in some form) for the other children to use during their

interactions. These clues could be in the form of natural clues (such as the shape,

texture, and weight of the design elements), or in the form of symbolic or Embodied

representations. These clues could also be in other forms such as facial expressions,

verbal directives, or in the change in tone of a child’s voice.

Once the children have perceived the clues offered by physical and perceived

affordances, they perform actions on the artefacts and the social world (that is, other

children and the environment) (Figure 30). Cooperation could be in the form of


physical support or intervention, or in the form of verbal dialogue and discussion.

Children use scaffolds to interact with the artefacts. These scaffolds could be tools

and processes that assist them to perform actions on the artefacts. For example, in

the game of Osmo, while trying to guide the balls onto the targets in the virtual

space, children placed objects on the sides of the physical space to block the balls

from escaping from the virtual space. They used A3 paper edges in the physical

space to guide balls onto the targets in the virtual space.

Children also used imitation strategies as scaffolds in intuitive interaction.

While playing with Monkey Blocks, for example, some saw the symmetry in the

layout and imitated the other child’s strategies to create the other half of the

symmetry. These strategies pertained to the choice of block, and its placement or

position. Similar imitative strategies were used in the physical Jenga game, where

children imitated strategies pertaining to the removal of blocks (e.g. push and pull),

and strategies to decide which block to remove (e.g. tapping on the blocks, and the

visual assessment of the preference for certain blocks). Cooperative activity and

scaffolding are thus presented in the action part of the children’s MEII (see Figure

30).

Figure 30 Actions side of the Model for Embodied Intuitive Interaction (MEII) in children.


In the course of children’s interactions with artefacts, they also act on the social

world (other children and the environment). These interactions could be a verbal

interaction with other children, or a physical interaction that changes the nature of

the environment (such as the structure of the physical Jenga stack, the structure of

the layout in Monkey Blocks, or the layout of the physical and virtual spaces in

Osmo).

The affordances offered by the artefacts determine actions that can be

performed on them. These actions, in turn, result in changes in the properties of the

artefacts and the social world. Any change in properties of the artefacts and the social

world results in a change in their affordances (Maier & Fadel, 2009). Children’s

interactions with the artefacts and the social world change, and this can further

change their properties. This cyclic phenomenon, referred to as ‘emergence’, is a

result of actions performed on the artefacts and the social world as the result of the

affordances they offer. This phenomenon is represented by the emergence layer in

Figure 30.

As artefacts and the social world evolve, children’s interactions also evolve.

Thus, through a continuous process of perception and action, children develop

knowledge and understanding of various elements of the artefacts and the social

world (Allen & Strathern, 2003). They learn about physical elements, knowledge of

other people, and the behaviour of the environment. Children refine their modes of

perception and their interpretation of what they perceive. They are then equipped

with a better means of sensing and perceiving the environment; this results in

changes to their internal representations and models. Thus, it can be said that

emergence results in changes in existing knowledge, or the development of new

knowledge in children. This process is represented by ‘Knowledge’ in Figure 30.

Thus, the continuous updating of knowledge that translates perceptions into response

is a property of a complex and dynamic Embodied intuitive system that constantly

evolves over time.

The MEII is a continuous cycle of perceive and act. Perception requires the use

of affordances through natural clues, Embodied representations and symbolic clues.

These affordances facilitate children’s decisions and actions on the artefact, using

scaffolding and co-operative activity. Emergence is the cyclic phenomenon that


occurs as the result of continuous action-perception iterations that result in change in

affordances and, thus, changes in knowledge.

8.3 SUMMARY

This chapter has linked the results and outcomes of Experiment 1 and

Experiment 2 to determine the study’s overall implications for design. Children’s

Embodied intuitive interactions with physical products, virtual interfaces, and TEIs

were discussed in relation to Blackler's (2008) continuum of intuitive interaction, and

the German-based Intuitive Use of User Interfaces (IUUI) research group’s

continuum of knowledge in intuitive interaction resulting in an Enhanced Framework

of Intuitive Interaction (EFII). Using the concept of dynamic anticipatory systems,

MEII - an interaction model that represents children’s Embodied intuitive

interactions was presented. The model represents children perceiving (Figure 29) and

acting (Figure 30) on artefacts and the social world (that is, other children and the

environment). The use of aspects of Embodiment in relation to the model is

discussed.

Perceiving and acting happens in a continuous cyclic manner through use of

aspects of Embodiment. This cyclic process of perceive and act ultimately results in

change in knowledge, and/or in the generation of new knowledge. Chapter 9

concludes this thesis with an overview of its research contributions, outcomes,

limitations, and the possibilities it suggests for future research.

Chapter 9: Contributions and Future Work

Children’s increasing use of complex products and concepts was a driving

force that motivated this study that investigated Embodied intuitive interaction in

children through the lens of human-centred and child-centred design, keeping their

particular experiences and knowledge in mind. Embodied intuitive interaction has

immense potential to offer the design of children’s products. This research

established four objectives to investigate its specific role in the design process. A


methodology that enabled a thorough elicitation of the specific aspects of

Embodiment that facilitate children’s intuitive interaction was developed. This

methodology comprised data collection that used observations, retrospective

interviews, and co-discovery methods.

Two experiments were carried out: Experiment 1 compared physical and

virtual products for Embodiment and intuitive interaction (Chapter 6), and

Experiment 2 determined the primary predictors of variability in children’s intuitive

interaction (Chapter 7). Chapter 8 discussed the findings of Experiment 1 and

Experiment 2, and their implications for design. The results were then discussed in

relation to the continua of intuitive interaction resulting in EFII - Enhanced

framework of intuitive interaction. MEII – a model for Embodied intuitive

interaction, based on dynamic anticipatory systems, was developed and discussed

(Section 8.2.2). MEII offers an understanding of how children perform Embodied

intuitive interactions with artefacts and the social world.

This research study pioneers the study of Embodiment and children’s intuitive

interaction, and extends the understanding of Embodied intuitive interaction to

interactions with physical products, virtual interfaces, and TEIs. These contributions

and outcomes are significant, not only in the context of children’s interaction, but

also (more generally) in the context of the study of design for Embodied intuitive

interaction.

Section 9.1 presents an overview of the study’s contributions to knowledge,

while 9.2 outlines the research outcomes. Limitations of the research are presented in

Section 9.3, and an overview of potential future research directions is presented in

Section 9.4. The chapter closes with the study’s conclusions in Section 9.5.

9.1 CONTRIBUTIONS TO KNOWLEDGE

This research has provided a number of significant contributions to knowledge.

It has generated new knowledge of, and insights into children’s Embodied intuitive

interaction. This new knowledge, in turn, contributes to the broader context of

children’s interaction with physical products, virtual interfaces, and TEIs.

Specifically, three original contributions to knowledge are outlined in this section: (i)

Knowledge of children’s Embodied intuitive interactions with tactile interactions as

an interaction modality; (ii) Understanding of the role of Embodiment in facilitating


children’s intuitive interaction; and (iii) A methodology for investigating children’s

Embodied intuitive interaction.

(i) Knowledge of children’s Embodied intuitive interactions with

tactile interactions as an interaction modality

This research study strengthens the understanding of Embodiment and

children’s intuitive interaction from the perspective of tactile interactions as an

interaction modality. Most previous research focussed on full body interactions

and Embodied metaphors as facilitators of intuitive interaction. Thus, this

research study provides new insights into the role of tactile interactions in

Embodied intuitive interaction with physical products, virtual interfaces, and

TEIs. This is a significant step towards incorporating Embodiment in the

design of children’s products on the physical-virtual continuum, as children are

familiar with tactile interactions from birth. Investigating Embodiment through

interaction modalities also opens up possibilities for the exploration of other

sensory-based interaction modalities (such as sound interactions).

(ii) An understanding of the role of Embodiment in facilitating

children’s intuitive interaction

The results from this study have contributed to an Enhanced Framework of

Intuitive Interaction (EFII) (Blackler et al., 2018), an adaptation of which a that

relates to children’s Embodied intuitive interaction was discussed in Section

8.1.2. This EFII could lead to new directions for research into intuitive

interaction, such as intuitive experiences, factors responsible for these

experiences and the characteristics of features (e.g. transfer distance,

indirection, ubiquity). The results from the study are transferrable to the EFII;

this transferability suggests that they could also be applicable to adults. This is

an important contribution because Embodied intuitive interaction is

conventionally associated with children as sensorimotor knowledge is

considered to be more accessible to children than to adults (Brandenburg &

Sachse, 2012). The applicability of the EFII (Figure 27) to adults could lead to

future research that investigates Embodied intuitive interaction in adults.

The EFII allows discussion of intuitive interaction with products on the

physical-virtual continuum: physical products, virtual interfaces, and TEIs. The


findings from the study concur with Blackler's (2008) findings that physical

affordances should be incorporated in design whenever possible. Where

physical affordances are not possible, perceived affordances should be used.

This research study suggests further implications for design, and provides

design recommendations for children’s Embodied intuitive interactions with

physical products, virtual interfaces, and TEIs (Section 7.3.1). Physical and

perceived affordances could be used together, be sequential in time, or/and

nested in space.

The research study has further highlighted the role of cooperative activity and

scaffolding as Embodied aspects in the design of children’s products.

Appropriate scaffolds can assist children in carrying out perception and action

processes. Emergence is a design aspect representative of dynamic processes in

systems where interactions, behaviours, and environments evolve over time.

Dynamic processes are conducive to Embodied intuitive interactions, as they

facilitate the updating of existing knowledge, and the generation of new

knowledge; in other words, the learning of new concepts. This contribution is

significant because efforts have not focussed in the past on designing dynamic

emergent systems for Embodied interactions. The study provides direction

future research and development in the form of possibilities of using

affordances to design emergent systems.

(iii) Methodology for investigating children’s Embodied intuitive

interaction

A robust research methodology suitable for the study of children’s Embodied

intuitive interaction was developed. The combination of methods used,

consisting of observations, co-discovery, and retrospective interviews proved

to be highly successful in capturing the complexity of children’s decision

making and intuitive and Embodied interactions. The significance of the

methodology was the participatory nature of observations, where children were

observed in natural conditions, playing with real toys, and with others whom

they knew before the experiment. The unobtrusive nature of the children’s

play, in which there was no intervention of any kind from the researcher,

ensured that there was minimal interference to the children’s cognitive

processes. This ensured that the aspects of Embodiment used by children in


intuitive interaction with the toys in the experiments were analogous to the

aspects used in everyday interactions. Coding schemes developed for

Embodiment and intuitive interaction enabled the recording of children’s

intuitive behaviours, and the identification of aspects of Embodiment in these

behaviours.

The coding of the raw data facilitated the use of methods that allowed analysis

of aspects of Embodiment that contribute to children’s intuitive interaction, and

the relationship between these aspects. These analyses enabled an

understanding of the use of aspects of Embodiment in children’s intuitive

interaction. Due to their effectiveness in the context of children’s play, it is

expected that the methods used in this research can be transferred to the study

of children’s Embodied intuitive interaction in other contexts, such as teaching

and learning (Gillies, 2016) and affective technology for children’s social

connectedness (Pallarino, Free, Mutuc, & Yarosh, 2016).

9.2 RESEARCH OUTCOMES

The primary outcome of this research study is MEII – an interaction model for

Embodied intuitive interaction for the design of children’s products (Figure 29 and

Figure 30). The findings suggest that children are distributed anticipatory systems,

relying on their sensorimotor knowledge, and using their sensory perceptual

structures to perceive and act on their physical and social worlds. MEII, therefore,

consists of children interacting with these physical and social worlds: the physical

world consisting of artefacts, and the social world consisting of other children,

adults, and the environment. Children use physical affordances, perceived

affordances, and scaffolding in cooperation with other children to perceive the clues

in the social world and the artefacts, to decide on the actions to be performed. Once

the decision is made, children perform actions on the artefacts and the social world.

Children use scaffolding in cooperation with other children to the perform

actions and, in the process, change the properties of the artefacts and the social

world. The artefacts, social world, and children’s interaction evolve over repeated

use of the product, and this evolution results in emergence. This process is

responsible for the dynamic nature of MEII which, in turn, is important for children’s

development of new knowledge, or the updating of their existing knowledge and


learning. This is a significant contribution as it offers insights into how children

interact with the physical and social world. MEII could be applied in various

contexts of children’s interactions in design, research, and development.

9.3 RESEARCH LIMITATIONS

The first limitation of the study is that children could be more familiar to

physical Jenga than the virtual Jenga. More familiar interface features are used more

intuitively (Blackler et al., 2010). Thus, children’s familiarity of physical Jenga could

have affected intuitive interaction in children. However, children’s familiarity to

physical Jenga represents experience and familiarity of children to physical objects,

physical interactions and material properties. Children could have played with

physical Jenga (thus making them more familiar with it) due to the intuitiveness of

the toy. The objective of Experiment 1 was to study what makes certain interfaces

more intuitive for children’s interaction and thus more familiar to them. A between

subjects comparative study between two groups of children, one playing with

physical Jenga and second group playing with virtual Jenga, was conducted. The

pairs were randomly allocated to each of the groups. This ensured that the familiarity

differences between the children participating in the two groups would balance each

other out (Charness, Gneezy, & Kuhn, 2012).

The second limitation of this research is that children were observed playing

with only one type of TEI, (towards the left of the physical virtual continuum). TEIs

differ depending on how the physical and virtual are configured and coupled. Due to

innovation in technology, new configurations of TEIs evolve such as overlapping

physical and virtual spaces (Ullmer & Ishii, 2000). Embodied intuitive interaction

could vary in different TEIs depending on how physical and virtual spaces are

configured. Investigating other TEI configurations could provide additional

guidelines for designing Embodied intuitive TEIs and could be the focus for future

research.

The third limitation of this research study is that the coding of the qualitative data

could have been influenced by researcher bias. To reduce this effect, however,

coding heuristics were developed for both Embodiment and intuitive interaction,

examples of which are given in Appendix D and Appendix E. Heuristics were

developed through a review of the literature on Embodiment and intuitive


interaction, and then finalised by applying the heuristics specific to Experiment 1 and

Experiment 2 to the raw data. Secondly, the same researcher coded all the data twice,

and performed reliability analysis on the two versions of the coding (as described in

Section 5.4.2).

The fourth limitation of this study was that the children might have been

conscious of being observed and video recorded. This could have compromised their

natural behaviour. This possibility was minimised, however, by observing children in

the context of play with another child. The children were paired for the experiment—

each child with another whom they already knew. The researcher played as the

children’s opposing ‘team’ in Experiment 1. The motivation to win the game, the

context of play, and the presence of another child whom they already knew, made the

children feel comfortable and at ease.

The fifth limitation of the study was presence of parents and teachers

influencing the behaviour of children during the game play. Observational studies in

intuitive interaction require design of studies that evoke natural behaviour in the

participants as much as possible. Children behave differently when around adults

(Gardner, 2000) and this could have affected the outcomes of the studies

(Experiment 1 and Experiment 2). Experiment 1 was thus conducted at a local state

school, in a classroom without a teacher. In Experiment 2 where the study was

conducted at People and Systems lab (PAS lab) at QUT, Parents were asked not to be

present during the study. However, some parents insisted that they wanted to be

present during the study. Some parents were inquisitive about the study itself,

specifically the toys used in the study. Some wanted to know what were the children

asked to do, how were their children performing in the study and some just wanted to

be around as they had young kids to look after. These parents were allowed to stay

back, but behind a two-way mirror so that children could not see their parents and

parents could not intervene or interfere with the study, but parents could see the

proceedings of the study.

Finally, some children, especially young children, got into an argument or fight

while playing. This could affect their game play and their interaction with the toys.

Arguments and fights to an extent where none of the children were emotionally or

physically harmed were allowed (part of cooperative activity). However, attempts

were made to mitigate the situation and when it went beyond control, the experiment


was stopped and discontinued. Data from such experiments were not analysed and

destroyed.

9.4 FUTURE RESEARCH

(i) Investigation into each of the aspects of Embodiment

This research study has identified the aspects of Embodiment—physical

affordances, perceived affordances, emergence, scaffolding, and cooperative

activity—from the literature on Embodied cognition and decision making.

Further research on individual aspects of Embodiment is now required.

Scaffolding, emergence, and cooperative activity were found to be strongly

correlated to physical affordances. Considering the pivotal role of physical

affordances in children’s Embodied intuitive interaction, it is imperative that

future research investigates ways to incorporate scaffolding, emergence, and

cooperative activity in products for children, using physical affordances.

(ii) Generalisation of the Model for Embodied intuitive interaction (MEII)

The model for Embodied intuitive interaction (MEII) offers an understanding

of children’s Embodied intuitive interaction with artefacts and the social world.

The social world in this research study was limited to two children cooperating

to play a game, and the environment in which the game was played. Researcher

participation in the cooperative play was included in Experiment 1. However,

the researcher’s role in the Embodied intuitive interaction was not considered

in the research analysis. Future research could focus on including a larger

number of children, children from other cultures, and adults (such as teachers

and parents) in the social world. This would provide more insight into the role

of social world in intuitive interaction and result in a more generalised model

for children’s Embodied intuitive interaction.

(iii) Broaden the scope of the research

The methods used in this research can also be applied to investigate children’s

Embodied intuitive interaction in other contexts, such as spaces and services.

Children’s use of space and the objects within them is a topic of interest for

interaction designers in gaming, and in the field of learning sciences. The

methods could also be applied to services for children in healthcare,


entertainment, and learning. Research on intuitive interaction has mostly

focussed on interactions with interfaces. As interfaces become increasing

incorporated with digital elements, for example TEIs and virtual interfaces,

they generate large amounts of digital data which offer limited possibilities of

interaction with them. Future research could investigate ways to interact with

the digital data intuitively with Embodied interactions. The methods used in

this study could be used to investigate ways to materialise the digital data.

9.5 CONCLUSIONS

This research study investigated the role of Embodiment in children’s intuitive

interaction. The novel approach used in the study elicited natural and reliable

children’s behaviour for Embodied intuitive interactions. The study has taken a

human-centred approach to understanding children’s Embodied intuitive interactions

as the focus of study. Addressing Embodiment from a design perspective, and

determining the design aspects of Embodiment, are novel initiatives towards

identifying the design elements that can facilitate children’s intuitive interaction.

The knowledge resulting from the research also provides a significant

contribution to the domain of child-product interaction. Of significance was the

investigation of the design aspects of Embodiment for intuitive interaction in three

types of children’s products: physical, virtual, and TEIs.

The extent to which the aspects of Embodiment explain the variability in

intuitive interaction was determined, and this allowed inferences to be made about

the contribution of each of the aspects of Embodiment to intuitive interaction. This

includes the notable role of physical affordances in Embodied intuitive interaction,

and the importance of both physical and perceived affordances in Embodied intuitive

interaction with TEIs.

The research has provided empirical evidence to support previous claims in the

literature that physical products are more intuitive than virtual interfaces. The study

further found that TEIs with both physical and virtual elements could be intuitive,

depending on how these elements are configured in the system. This is an important

finding because it allows for the design of children’s products on the physical-virtual

continuum, with variations of physical and virtual compositions. To date, this has

been considered a challenging feat.


Research findings have resulted in the development of MEII for children. MEII

provides a representation of the use of aspects of Embodiment in perception and

action processes that form the basis of any anticipatory system, such as children.

Designers could use MEII as a tool to evaluate and design Embodied intuitive

products for children. Since MEII is based on children’s everyday behaviour and

interactions in the context of play, it could be transferred to other contexts involving

children. Based on its results and findings, the study has offered design

recommendations that are applicable to three main types of intuitive products for

children: physical, virtual, and TEIs, the most prominent types of products currently

used by children.

The new knowledge generated by this research has implications for the design

of a wide range of children’s products, such as physical products, virtual interfaces,

and TEIs. In particular, recommendations from this research are directed at

informing the design of interfaces for children that incorporate Embodiment to

facilitate intuitive interactions. Rather than focusing on possible technological

innovations, the study has taken a human-centred perspective. Its recommendations

thus aim to improve children’s experience in interacting with products and interfaces.

It is expected that the design guidelines will support the development of relevant

sensory perceptual structures, and the use of relevant knowledge and experiences that

is required for Embodied intuitive interaction with interfaces.

Pioneering the study of children’s Embodied intuitive interaction using play as

a context of study, this research contributes a novel perspective on children’s

interactions with the world of artefacts. It is therefore expected that the new

knowledge developed will stimulate discussions in diverse application domains

involving children.

Finally, this study has consolidated various aspects of Embodiment and children’s

intuitive interaction. It has thus made a significant contribution to the study of

Embodiment and children’s intuitive interaction, and advanced the understanding of

how children interact with artefacts.

Bibliography 183

Bibliography

Adebesin, T. F., De Villiers, M. R., & Ssemugabi, S. (2009). Usability testing of e-learning: an approach incorporating co-discovery and think-aloud. In Proceedings of the 2009 Annual Conference of the Southern African Computer Lecturers’ Association (pp. 6–15).

Ahmet, Z., Jonsson, M., Sumon, S. I., & Holmquist, L. E. (2011). Supporting embodied exploration of physical concepts in mixed digital and physical interactive settings. In Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction (pp. 109–116).

Alexander, G. M. (2003). An evolutionary perspective of sex-typed toy preferences: Pink, blue, and the brain. Archives of Sexual Behavior, 32(1), 7–14.

Allen, P. M., & Strathern, M. (2003). Evolution, emergence, and learning in complex systems. Emergence, 5(4), 8–33.

Als, B. S., Jensen, J. J., & Skov, M. B. (2005). Comparison of think-aloud and constructive interaction in usability testing with children. Interaction Design and Children, 9–16. http://doi.org/10.1145/1109540.1109542

Alter, A. L., Oppenheimer, D. M., Epley, N., & Eyre, R. N. (2007). Overcoming intuition: metacognitive difficulty activates analytic reasoning. Journal of Experimental Psychology: General, 136(4), 569.

Amazon, & Foxconn. (2007). Kindle.

Andersen, P. B. (2001). What semiotics can and cannot do for HCI. Knowledge-Based Systems, 14(8), 419–424.

Anderson, M. (2003). Embodied Cognition: A field guide. Artificial Intelligence, 149(1), 91–130. http://doi.org/http://dx.doi.org/10.1016/S0004-3702(03)00054-7

Anderson, M. (2005). How to study the mind: An introduction to embodied cognition. Embodied Cognition and Perceptual Learning in …. Retrieved from http://cogprints.org/3945/

Anderson, S. (2011). Seductive interaction design: creating playful, fun, and effective user experiences. Pearson Education.

Anthony, L., Stofer, K. A., Luc, A., & Wobbrock, J. O. (2016). Gestures by Children and Adults on Touch Tables and Touch Walls in a Public Science Center. In Proceedings of the The 15th International Conference on Interaction Design and Children (pp. 344–355).

Antle, A. (2007). Designing tangibles for children: what designers need to know. In CHI’07 Extended Abstracts on Human Factors in Computing Systems (pp. 2243–2248).

184 Bibliography

Antle, A. (2011). Exploring how children use their hands to think: an embodied interactional analysis. Behaviour & Information Technology, 1–17. http://doi.org/10.1080/0144929X.2011.630415

Antle, A., Corness, G., & Droumeva, M. (2009a). Human-computer-intuition? Exploring the cognitive basis for intuition in embodied interaction. International Journal of Arts and Technology, 2(3), 235. http://doi.org/10.1504/IJART.2009.028927

Antle, A., Corness, G., & Droumeva, M. (2009b). What the body knows: Exploring the benefits of embodied metaphors in hybrid physical digital environments. Interacting with Computers, 21(1–2), 66–75. http://doi.org/10.1016/j.intcom.2008.10.005

Antle, A., Droumeva, M., & Corness, G. (2008). Playing with the sound maker: do embodied metaphors help children learn? … on Interaction Design and Children, 178–185. Retrieved from http://dl.acm.org/citation.cfm?id=1463754

Antle, A., & Wise, A. (2013). Getting down to details: Using theories of cognition and learning to inform tangible user interface design. Interacting with Computers, 25(1), 1–40. Retrieved from http://iwc.oxfordjournals.org/content/25/1/1.short

Apple. (2001). iTunes.

Apple. (2015). iPad.

Arthur, C. (2010, June). Why Minority Report was spot on. The Guardian. Retrieved from https://www.theguardian.com/technology/2010/jun/16/minority-report-technology-comes-true

Aslan, I., Primessnig, F., Murer, M., Moser, C., & Tscheligi, M. (2013). Inspirations from honey bees: exploring movement measures for dynamic whole body gestures. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces (pp. 421–424).

Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., & MacIntyre, B. (2001). Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21(6), 34–47. http://doi.org/10.1109/38.963459

Baars, B. J., Ramsøy, T. Z., & Laureys, S. (2003). Brain, conscious experience and the observing self. Trends in Neurosciences, 26(12), 671–675.

Baillargeon, R., & Graber, M. (1988). Evidence of location memory in 8-month-old infants in a nonsearch AB task. Developmental Psychology, 24(4), 502.

Bakker, S., Van Den Hoven, E., & Antle, A. N. (2011). MoSo tangibles: evaluating embodied learning. In Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction (pp. 85–92).

Barrett, L. (2011). Beyond the brain: How body and environment shape animal and human minds. Princeton University Press.

Bibliography 185

Barsalou, L. W. (2010). Grounded Cognition: Past, Present, and Future. Topics in Cognitive Science, 2(4), 716–724. http://doi.org/10.1111/j.1756-8765.2010.01115.x

Barsalou, L. W., Niedenthal, P. M., Barbey, A. K., & Ruppert, J. A. (2003). Social embodiment. Psychology of Learning and Motivation, 43, 43–92.

Barsalou, L. W., Santos, A., Simmons, W. K., & Wilson, C. D. (2008). Language and simulation in conceptual processing. In Symbols, embodiment, and meaning (pp. 245–283).

Bastick, T. (1982). Intuition: How we think and act.

Bastick, T. (2003). Intuition: Evaluating the construct and its impact on creative thinking. Stoneman & Lang.

Beaudouin-Lafon, M. (2000). Instrumental interaction: an interaction model for designing post-WIMP user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 446–453).

Bingham, G. P. (1995). Dynamics and the problem of visual event recognition. Mind as Motion: Explorations in the Dynamics of Cognition, 403–448.

Björk-Willén, P., & Cromdal, J. (2009). When education seeps into “free play”: How preschool children accomplish multilingual education. Journal of Pragmatics, 41(8), 1493–1518.

Blackler, A. (2008). Intuitive interaction with complex artefacts : empirically-based research. VDM Verlag, Saarbrücken, Germany.

Blackler, A., Desai, S., McEwan, M., Dieffenbach, S., & Popovic, V. (2018). Perspectives on the nature of intuitive interaction. In Blackler (Ed.), Intuitive Interaction: Research and Application. CRC Press.

Blackler, A., & Hurtienne, J. (2007). Towards a unified view of intuitive interaction : definitions, models and tools across the world. MMI-Interaktiv, 13(2007), 36–54. Retrieved from http://eprints.qut.edu.au/19116/

Blackler, A., & Popovic, V. (2015). Towards intuitive interaction theory. Interacting with Computers, 27(3), 203–209.

Blackler, A., & Popovic, V. (2016). Intuitive Interaction research – new directions and possible responses. In 2016 Design Research Society 50th Anniversary Conference (pp. 1–11). Brighton.

Blackler, A., Popovic, V., & Mahar, D. (2010). Investigating users’ intuitive interaction with complex artefacts. Applied Ergonomics, 41(1), 72–92. http://doi.org/10.1016/j.apergo.2009.04.010

Blikstein, P. (2013). Digital Fabrication and ’Making’ in Education: The Democratization of Invention. In J. W.-H. Büching (Ed.), FabLabs: Of Machines, Makers and Inventors. Bielefeld: Transcript Publishers.

186 Bibliography

Brandenburg, S., & Sachse, K. (2012). Intuition comes with experience. In Human factors: A view from an integrative perspective. Proceedings of the Human Factors and Ergonomics Society of Europe Conference (pp. 213–223).

Breathnach, H., O’Gorman, L. M., & Danby, S. (2016). “Well it depends on what you’d call play”: Parent perspectives on play in Queensland’s Preparatory Year. Australasian Journal of Early Childhood, 41(2), 77–84.

Brederode, B., Markopoulos, P., Gielen, M., Vermeeren, A., & de Ridder, H. (2005). pOwerball: the design of a novel mixed-reality game for children with mixed abilities. Interaction Design and Children, 32–39. http://doi.org/10.1145/1109540.1109545

Brooks, R. A. (1990). Elephants don’t play chess. Robotics and Autonomous Systems, 6(1), 3–15.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42.

Bruner, J. S., Jolly, A., & Sylva, K. (1976). Play: Its role in development and evolution.

Buur, J., & Andreasen, M. M. (1989). Design models in mechatronic product development. Design Studies, 10(3), 155–162.

Caldera, Y. M., Huston, A. C., & O’Brien, M. (1989). Social interactions and play patterns of parents and toddlers with feminine, masculine, and neutral toys. Child Development, 70–76.

Cave, A., Blackler, A., Popovic, V., & Kraal, B. (2014). Examining Intuitive Navigation in Airports. In Design Research So- ciety Conference 2014. Umea, Sweden.

Chalmers, D. J. (1995). Facing up to the problem of consciousness. Journal of Consciousness Studies, 2(3), 200–219.

Chalmers, M. (2001). Book Review: Where the Action Is: The Foundations of Embodied Interaction, Paul Dourish. MIT Press, Cambridge.

Charness, G., Gneezy, U., & Kuhn, M. A. (2012). Experimental methods: Between-subject and within-subject design. Journal of Economic Behavior & Organization, 81(1), 1–8.

Chiel, H. J., & Beer, R. D. (1997). The brain has a body: adaptive behavior emerges from interactions of nervous system, body and environment. Trends in Neurosciences, 20(12), 553–557.

Chin, J., Diehl, V., & Norman, K. (1988). Development of an instrument measuring user satisfaction of the human-computer interface. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 213–218).

Choi, K.-S. (1993). Developmental changes in intuitive thinking of Korean children. Early Child Development and Care, 85(1), 89–95.

Bibliography 187

Churchman, C. W. (1971). The Design of Inquiring Systems Basic Concepts of Systems and Organization.

Cienki, A., & Müller, C. (2008). Metaphor and gesture (Vol. 3). John Benjamins Publishing.

Clark, A. (1997). Being there: Putting brain, body, and world together again. Being There: Putting Brain, Body, and World Together Again. Cambridge, MA: MIT Press.

Clark, A. (2001). Reasons, Robots and the Extended Mind. Mind & Language, 16(2), 121–145. http://doi.org/10.1111/1468-0017.00162

Clark, A. (2005). Beyond the flesh: Some lessons from a mole cricket. Artificial Life, 11(1–2), 233–244.

Clark, A. (2008). Supersizing the Mind: Embodiment, Action, and Cognitive Extension. Oxford University Press. Retrieved from http://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780195333213.001.0001/acprof-9780195333213-chapter-1.

Clark, A. (2012). The Cambridge Handbook of Cognitive Science. In K. Ramsey & F. and William (Eds.), The Cambridge Handbook of Cognitive Science (pp. 275–291). Cambridge University Press. http://doi.org/http://dx.doi.org/10.1017/CBO9781139033916.018

Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. The Behavioral and Brain Sciences, 36(3), 181–204. http://doi.org/10.1017/S0140525X12000477

Clement, J. (1994). Use of physical intuition and imagistic simulation in expert problem solving.

Cohen, J. (1992). A power primer. Psychological bulletin (Vol. 112). American Psychological Association.

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2013). Applied multiple regression/correlation analysis for the behavioral sciences. Routledge.

Cole, M., & Griffin, P. (1980). Cultural amplifiers reconsidered. The Social Foundations of Language and Thought, Essays in Honor of Jerome S. Bruner, 343–364.

Corr, P. J. (2008). The reinforcement sensitivity theory of personality. Cambridge University Press.

Cowart, M. (2004). Embodied cognition. The Internet Encyclopedia of Philosophy.

Crowle, S., Boniface, M., Poussard, B., & Asteriadis, S. (2014). A design and evaluation framework for a tele-immersive mixed reality platform. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8853, 151–158. http://doi.org/10.1007/978-3-319-13969-2_12

188 Bibliography

Cruise, T., Wagner, P., & Brian, D. P. (1996). Mission Impossible. United States: Paramount Pictures.

Cutting, J., & Vishton, P. (1995). Perceiving layout and knowing distances: the integration relative potency and contextual use of different information about depth epstein w. rogers s. perception of space and motion 1995 69--117. Academic Press, San Diego, CA.

Dahl, D. (2017). Multimodal Interaction with W3C Standards: Toward Natural User Interfaces to Everything. Springer.

Dane, E., & Pratt, M. G. (2007). Exploring intuition and its role in managerial decision making. Academy of Management Review, 32(1), 33–54. article.

Dautenhahn, K. (1996). Embodied cognition in animals and artifacts. Proc. AAAI FS Embodied Cognition and Action, 27–32.

Dennett, D. (1991). Consciousness explained. New York: Little Brown & Co.

Desai, S., Blackler, A., & Popovic, V. (2015). Intuitive use of tangibles. In IASDR.

Desai, S., Blackler, A., & Popovic, V. (2016). Intuitive interaction in a mixed reality system (pp. 1–16). DRS.

Dick, P. K. (2002). The Minority Report: And Other Classic Stories (Vol. 4). Citadel Press.

Diefenbach, S., & Ullrich, D. (2015). An Experience Perspective on Intuitive Interaction : Central Components and the Special Effect of Domain Transfer Distance. Interacting with Computers, 27(3), 210–234. http://doi.org/10.1093/iwc/iwv001

Disessa, A. A. (1988). Knowledge in pieces.

Dix, A. (2011). Physical creatures in a digital world. In Proceedings of the 29th Annual European Conference on Cognitive Ergonomics (pp. 11–14).

Djajadiningrat, T., Wensveen, S., Frens, J., & Overbeeke, K. (2004). Tangible products: redressing the balance between appearance and action. Personal and Ubiquitous Computing, 8(5), 294–309.

Dotov, D. G., Nie, L., & De Wit, M. M. (2012). Understanding affordances: history and contemporary. Choices, 33, 269–298.

Dourish, P. (1927). Seeking a foundation for context-aware computing. Human--Computer Interaction, 16(2–4), 229–241.

Dourish, P. (2001). Where the action is: the foundations of embodied interaction. The MIT Press.

Draper, N., & Smith, H. (2014). Applied regression analysis (Third). John Wiley & Sons, Ltd.

Dreyfus, H., & Dreyfus, S. E. (2000). Mind over machine. Free Press.

Druin, A. (2002). The role of children in the design of new technology. Behaviour and Information Technology, 21(1), 1–25.

Bibliography 189

Dunn, B. D., Galton, H. C., Morgan, R., Evans, D., Oliver, C., Meyer, M., … Dalgleish, T. (2010). Listening to your heart how interoception shapes emotion experience and intuitive decision making. Psychological Science. article.

Eelen, J., Dewitte, S., & Warlop, L. (2013). Situated embodied cognition: Monitoring orientation cues affects product evaluation and choice. Journal of Consumer Psychology.

Ekman, P. (2006). Darwin and facial expression: A century of research in review. Ishk.

Ekman, P., Friesen, W. V, & Ellsworth, P. (2013). Emotion in the human face: Guidelines for research and an integration of findings. Elsevier.

Engeström, Y., Miettinen, R., & Punamäki, R.-L. (1999). Perspectives on activity theory. Cambridge University Press.

Entwistle, J. (2015). The Fashioned Body: Fashion, Dress and Social Theory. John Wiley & Sons.

Enyedy, N., Danish, J., Deliema, D., Saleh, A., Lee, C., Morris, N., & Illum, R. (2017). Social Affordances of Mixed Reality Learning Environments: A case from the Science through Technology Enhanced Play project (STEP). In Proceedings of the 50th Hawaii International Conference on System Sciences.

Eriksen, T. H. (2001). Small Places, Large Issues-: An Introduction to Social and Cultural Anthropolog. Pluto Press.

Erzberger, C., & Kelle, U. (2003). Making inferences in mixed methods: the rules of integration. Handbook of Mixed Methods in Social and Behavioral Research, 457–490.

Fan, M., Antle, A. N., & Cramer, E. S. (2016). Exploring the Design Space of Tangible Systems Supported for Early Reading Acquisition in Children with Dyslexia. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 689–692).

Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160.

Feinstein, N. W., & Meshoulam, D. (2013). Science for what public? Addressing equity in American science museums and science centers. Journal of Research in Science Teaching.

Felemban, S., Gardner, M., Callaghan, V., & Pena-Rios, A. (2017). Towards Observing and Assessing Collaborative Learning Activities in Immersive Environments. In International Conference on Immersive Learning (pp. 47–59).

Fernaeus, Y., & Jacobsson, M. (2009). Comics, robots, fashion and programming: outlining the concept of actDresses. In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (pp. 3–8). New York, NY, USA: ACM. http://doi.org/10.1145/1517664.1517669

190 Bibliography

Field, A. (2008). Discovering statistics using SPSS, and sex, drugs and rock “n”roll. SAGE Publications Ltd, ISBN.

Fincham, B. (2016). Theorising Fun. In The Sociology of Fun (pp. 27–46). London: Springer. http://doi.org/10.1057/978-1-137-31579-3_2

Fink, J., Lemaignan, S., Dillenbourg, P., Rétornaz, P., Vaussard, F., Berthoud, A., … Franinović, K. (2014). Which robot behavior can motivate children to tidy up their toys?: Design and evaluation of ranger. In Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction (pp. 439–446).

Finucane, M. L., Alhakami, A., Slovic, P., & Johnson, S. M. (2000). The affect heuristic in judgments of risks and benefits. Journal of Behavioral Decision Making, 13(1), 1. article.

Fischbein, E. (1987). Intuition in science and mathematics: An educational approach (Vol. 5). Springer.

Fischbein, E. (1999). INTUITIONS AND SCHEMATA IN MATHEMATICAL REASONING, (1968), 11–50.

Fischer, J., Jiang, W., Kerne, A., Greenhalgh, C., Ramchurn, S., Reece, S., … Rodden, T. (2014). Supporting Team Coordination on the Ground : Requirements from a Mixed Reality Game. COOP 2014 - Proceedings of the 11th International Conference on the Design of Cooperative Systems, 49–67. http://doi.org/10.1007/978-3-319-06498-7_4

Fischer, S., Itoh, M., & Inagaki, T. (2015). Screening Prototype Features in Terms of Intuitive Use: Design Considerations and Proof of Concept. Interacting with Computers, 27(3), 256–270. http://doi.org/10.1093/iwc/iwv002

Fisk, A. D., Rogers, W. A., Charness, N., Czaja, S. J., & Sharit, J. (2009). Designing for older adults: Principles and creative human factors approaches. CRC press.

Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6), 381.

Fitzmaurice, G. W., Ishii, H., & Buxton, W. A. S. (1995). Bricks: laying the foundations for graspable user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 442–449).

Forgas, J. P. (1995). Mood and judgment: the affect infusion model (AIM). Psychological Bulletin, 117(1), 39. article.

Forrester, J. W. (1995). The beginning of system dynamics. McKinsey Quarterly, (4), 4–16. http://doi.org/10.3401/poms.1080.0022

Franinović, K., & Serafin, S. (2013). Sonic interaction design. Mit Press.

Frederking, J., Cruz, M., Overbeeke, K., & Baskinger, M. (2007). Collaborative Play Through Digital and Physical Interaction. Tangible Play: Research and Design and Tangible and Tabletop Games at IUI, 7. Retrieved from http://elisevandenhoven.com/publications/hoven-iui07wp.pdf#page=33

Bibliography 191

Gaines, B. R. (1989). Social and cognitive processes in knowledge acquisition. Knowledge Acquisition, 1(1), 39–58. http://doi.org/10.1016/S1042-8143(89)80004-4

Gaines, B. R. (2013). Knowledge acquisition: Past, present and future. International Journal of Human-Computer Studies, 71(2), 135–156. http://doi.org/10.1016/j.ijhcs.2012.10.010

Gallese, V., & Lakoff, G. (2005). The brain’s concepts: The role of the sensory-motor system in conceptual knowledge. Cognitive Neuropsychology, 22(3), 455–479.

Gardner, F. (2000). Methodological issues in the direct observation of parent--child interaction: Do observational findings reflect the natural behavior of participants? Clinical Child and Family Psychology Review, 3(3), 185–198.

Gardner, M., & Elliott, J. B. (2014). The Immersive Education Laboratory : understanding affordances , structuring experiences , and creating constructivist, collaborative processes , in mixed-reality smart environments. Transactions on Future Intelligent Educational Environments, 1(1), 1–13.

Gaver, W. W. (1991). Technology affordances. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 79–84).

Gervais, R., Frey, J., Gay, A., Lotte, F., & Hachet, M. (2016). Tobe: Tangible out-of-body experience. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 227–235).

Gibbs, R. W. (2006). Embodiment and cognitive science. Cambridge University Press.

Gibson. (1979). The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum.

Gibson. (2014). The ecological approach to visual perception: classic edition. Psychology Press. (Originally published in 1979)

Gibson, E. (1988). Exploratory Behavior In The Development Of Perceiving, Acting, And The Acquiring Of Knowledge. Annual Review of Psychology, 39(1), 1–41. http://doi.org/10.1146/annurev.psych.39.1.1

Gillies, R. M. (2016). Cooperative learning: Review of research and practice. Australian Journal of Teacher Education, 41(3), 3.

Giummarra, M. J., Gibson, S. J., Georgiou-Karistianis, N., & Bradshaw, J. L. (2008). Mechanisms underlying embodiment, disembodiment and loss of embodiment. Neuroscience & Biobehavioral Reviews, 32(1), 143–160.

Goldman, A., & de Vignemont, F. (2009). Is social cognition embodied? Trends in Cognitive Sciences, 13(4), 154–159.

Goodwin, C. (2000). Action and embodiment within situated human interaction. Journal of Pragmatics, 32(10), 1489–1522.

192 Bibliography

Gopnik, A., Meltzoff, A. N., & Kuhl, P. K. (2009). The scientist in the crib: Minds, brains, and how children learn. HarperCollins.

Gore, J., & Sadler-Smith, E. (2011). Unpacking intuition: A process and outcome framework. Review of General Psychology, 15(4), 304.

Gorriz, C. M., & Medina, C. (2000). Engaging Girls with Computers Through Software Games. Commun. ACM, 43(1), 42–49. http://doi.org/10.1145/323830.323843

Gray, C. D., & Kinnear, P. R. (2012). IBM SPSS statistics 19 made simple. Psychology Press.

Grenier, A. M. (2007). Crossing age and generational boundaries: Exploring intergenerational research encounters. Journal of Social Issues, 63(4), 713–727.

Grufberg, K., & Jonsson, M. (2012). Sciensations: making sense of science by designing with sensors. In Proceedings of the 11th International Conference on Interaction Design and Children (pp. 116–124).

Guha, M. L., Druin, A., & Fails, J. A. (2013). Cooperative Inquiry revisited: Reflections of the past and guidelines for the future of intergenerational co-design. International Journal of Child-Computer Interaction, 1(1), 14–23. http://doi.org/10.1016/j.ijcci.2012.08.003

Haans, A., & IJsselsteijn, W. (2006). Mediated social touch: A review of current research and future directions. Virtual Reality, 9(2–3), 149–159. http://doi.org/10.1007/s10055-005-0014-2

Hammond, K. R. (1993). Naturalistic decision making from a Brunswikian viewpoint: Its past, present, future. Decision Making in Action: Models and Methods, 205–227.

Harnad, S. (2003). Symbol-grounding Problem. Encyclopedia of Cognitive Science.

Harrison, C., & Hudson, S. E. (2009). Providing Dynamically Changeable Physical Buttons on a Visual Display. Proceedings of the 27th International Conference on Human Factors in Computing Systems - CHI 09, 299–308. http://doi.org/10.1145/1518701.1518749

Hartson, H. R., Andre, T. S., & Williges, R. C. (2003). Criteria for evaluating usability evaluation methods. International Journal of Human-Computer Interaction, 15(1), 145–181.

Hasbro, & Scott, L. (2001). Jenga Blocks. Parker Brothers, Hasbro.

Haugeland, J. (1989). Artificial intelligence: The very idea. The MIT Press.

Haugeland, J. (1998). Mind embodied and embedded. Having Thought: Essays in the Metaphysics of Mind, 207–237.

Hay, S., Newman, J., & Harle, R. (2008). Optical tracking using commodity hardware. In Mixed and Augmented Reality, 2008. ISMAR 2008. 7th IEEE/ACM International Symposium on (pp. 159–160).

Bibliography 193

Hayles, N. K. (2010). My mother was a computer: digital subjects and literary texts. University of Chicago Press.

Heath, S. (2002). Embedded systems design. Newnes.

Hertzum, M. (2016). Usability Testing : Too Early ? Too Much Talking ? Too Many Problems ? Journal of Usability Studies (JUS), 11(3), 83–88.

Hespanhol, L., & Tomitsch, M. (2015). Strategies for intuitive interaction in public urban spaces. Interacting with Computers, iwu051.

Heyns, B. (1987). Schooling and cognitive development: Is there a season for learning? Child Development, 1151–1160.

Hinton, A. (2014). Perception, Cognition, and Affordance. In Understanding context: environment, language, and information architecture. “ O’Reilly Media, Inc.”

Hoffmann, M., Schuster, K., Schilberg, D., & Jeschke, S. (2016). Next-generation teaching and learning using the virtual theatre. In Automation, Communication and Cybernetics in Science and Engineering 2015/2016 (pp. 281–291). Springer.

Holmquist, L. E., Ju, W., Jonsson, M., Tholander, J., Ahmet, Z., Sumon, S. I., … Winograd, T. (2010). Wii Science: Teaching the laws of nature with physically engaging video game technologies. CHI 2010 Workshop: Video Games as Research Instruments, Altlanta.

Holyoak, K. J. (1991). 12 Symbolic connectionism: toward third-generation theories of expertise. Toward a General Theory of Expertise: Prospects and Limits, 301.

Hornecker, E., & Buur, J. (2006). Getting a grip on tangible interaction: a framework on physical space and social interaction. In Proceedings of the SIGCHI conference on Human Factors in computing systems (pp. 437–446). http://doi.org/http://doi.acm.org/10.1145/1124772.1124838

Hostetter, A. B., & Alibali, M. W. (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review, 15(3), 495–514.

Höysniemi, J., Hämäläinen, P., & Turkki, L. (2003). Using peer tutoring in evaluating the usability of a physically interactive computer game with children. Interacting with Computers, 15(2 SPEC.), 203–225. http://doi.org/10.1016/S0953-5438(03)00008-0

Hulin, W. S., & Katz, D. (1935). The Frois-Wittmann pictures of facial expression. Journal of Experimental Psychology, 18(4), 482.

Hummels, C., Smets, G., & Overbeeke, K. (1998). An intuitive two-handed gestural interface for computer supported product design. … Sign Language in Human-Computer …. Retrieved from http://link.springer.com/chapter/10.1007/BFb0053000

Hurtienne, J. (2007). Metaphors as Tools for Intuitive Interaction with Technology. Metaphorik. de, 12(2), 21–52.

Hurtienne, J. (2009). Image schemas and design for intuitive use.

194 Bibliography

Hurtienne, J., & Israel, J. H. J. J. H. (2007). Image schemas and their metaphorical extensions: intuitive patterns for tangible interaction. … on Tangible and Embedded Interaction, 15–17. http://doi.org/http://doi.acm.org/10.1145/1226969.1226996

Hurtienne, J., Klöckner, K., Diefenbach, S., Nass, C., & Maier, A. (2015). Designing with image schemas: resolving the tension between innovation, inclusion and intuitive use. Interacting with Computers, 27(3), 235–255.

Hurtienne, J., Löffler, D., Gadegast, P., & Hußlein, S. (2015). Comparing Pictorial and Tangible Notations of Force Image Schemas (pp. 249–256).

Hurtienne, & Blessing. (2007). Design for Intuitive Use-Testing image schema theory for user interface design. … Conference on Engineering Design, (August), 1–12. Retrieved from http://joernhurtienne.com/Publications_files/Paper_386_HurtienneBlessing.pdf

Husserl, E. (2013). Cartesian meditations: An introduction to phenomenology. Springer Science & Business Media.

Hutchins, E. (2000). Distributed cognition. Internacional Enciclopedia of the Social and Behavioral Sciences.

Ishii, H. (2007). Tangible user interfaces. In A. Sears & J. Jacko (Eds.), Human-Computer Interaction: Design Issues, Solutions, and Applications (pp. 141–157). CRC Press.

Ishii, H. (2008). Tangible bits: beyond pixels. In Proceedings of the 2nd international conference on Tangible and embedded interaction (pp. xv--xxv).

Israel, J. H., Hurtienne, J., Pohlmeyer, A. E., Mohs, C., Kindsmüller, M. C., & Naumann, A. (2009). On intuitive use , physicality and tangible user interfaces Jörn Hurtienne Anja Naumann, 2(4), 348–366.

Israel, J. H., Hurtienne, J., Pohlmeyer, A. E., Mohs, C., Kindsmuller, M., & Naumann, A. (2009). On intuitive use, physicality and tangible user interfaces. International Journal of Arts and Technology, 2(4), 348–366. http://doi.org/10.1504/IJART.2009.02924

Iverson, J. M., & Thelen, E. (1999). Hand, mouth and brain. The dynamic emergence of speech and gesture. Journal of Consciousness Studies, 6(11–12), 11–12.

Jacob, R. J. K., Girouard, A., Hirshfield, L. M., Horn, M. S., Shaer, O., Solovey, E. T., & Zigelbaum, J. (2008). Reality-based interaction: a framework for post-WIMP interfaces. In Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (pp. 201–210). ACM. http://doi.org/10.1145/1357054.1357089

Jacoby, S., Gutwillig, G., Jacoby, D., Josman, N., Weiss, P. L., Koike, M., … Sharlin, E. (2009). PlayCubes: monitoring constructional ability in children using a tangible user interface and a playful virtual environment. In Virtual Rehabilitation International Conference, 2009 (pp. 42–49).

Jenkins, R. (2014). Social identity. Routledge.

Bibliography 195

Johansson, C., Zeynep, A., Tholander, J., Jonsson, M., Islam Sumon, S., & Aleo, F. (2011). Weather Gods and Fruit Kids--embodying abstract concepts using tactile feedback and full body interaction. In CSCL 2011.

Johnson, D. W., & Johnson, R. T. (1994). Learning together and alone. Cooperative, competitive, and individualistic learning. ERIC.

Johnson, E., & Russo, E. (1981). Product familiarity and learning new information. Advances in Consumer Research, 8(1).

Johnson, M. (1987). The body in the mind: The bodily basis of meaning, imagination, and reason.

Johnson, M. (2010). Metaphor and Cognition. In D. Schmicking & S. Gallagher (Eds.), Handbook of Phenomenology and Cognitive Science (pp. 401–414). Dordrecht: Springer Netherlands. http://doi.org/10.1007/978-90-481-2646-0_22

Jones, L., & Lederman, S. (2006). Human hand function. Oxford University Press.

Jones, N. G. B. (1971). Criteria for use in describing facial expressions of children. Human Biology, 365–413.

Jordà, S., Geiger, G., Alonso, M., & Kaltenbrunner, M. (2007). The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces. In Proceedings of the 1st international conference on Tangible and embedded interaction (pp. 139–146).

Kahler, H., Kensing, F., & Muller, M. (2000). Methods \& tools: constructive interaction and collaborative work: introducing a method for testing collaborative systems. Interactions, 7(3), 27–34.

Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: a failure to disagree. American Psychologist, 64(6), 515.

Kail, R. (2012). Children and Their Development (6th ed.). Pearson.

Kemp, J. A. M., & Van Gelderen, T. (1996). Co-discovery exploration: an informal method for the iterative design of consumer products. Usability Evaluation in Industry, 139–146.

Kim, Y. S. (2015). A methodology of design for affordances using affordance feature repositories. AI EDAM, 29(Special Issue 03), 307–323. http://doi.org/10.1017/S0890060415000281

King, S., & Chang, K. (2016). Understanding Industrial Design: Principles for UX and Interaction Design. “ O’Reilly Media, Inc.”

Kirsh, D. (2013a). Embodied cognition and the magical future of interaction design. ACM Transactions on Computer-Human Interaction (TOCHI), 20(1), 3.

Kirsh, D. (2013b). Thinking with external representations. In Cognition Beyond the Brain (pp. 171–194). Springer.

Kirsh, D., & Maglio, P. (1994). On distinguishing epistemic from pragmatic action. Cognitive Science, 18(4), 513–549.

196 Bibliography

Kiverstein, J. (2010). Sensorimotor knowledge and the contents of experience. Perception, Action, and Consciousness, 257–274.

Klein, G. A. (2003). Intuition at work: Why developing your gut instincts will make you better at what you do. Currency/Doubleday.

Koo, T., & Li, M. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163.

Kutner, M. H., Nachtsheim, C., & Neter, J. (2004). Applied linear regression models. McGraw-Hill/Irwin.

Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: U of Chicago P. Chicago, Illinois, USA: Univ. of Chicago Press.

Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh: The embodied mind and its challenge to western thought. Basic books.

Lawry, S., Popovic, V., & Blackler, A. L. (2011). Diversity in product familiarity across younger and older adults. In Diversity \& Unity, 4th World Conference on Design Research, IASDR2011.

Leakey, R. (2008). The origin of humankind. Basic books.

Liamputtong, P. (2006). Researching the vulnerable: A guide to sensitive research methods. Sage.

Lim, K. H., Ward, L. M., & Benbasat, I. (1997). An empirical study of computer system learning: Comparison of co-discovery and self-discovery methods. Information Systems Research, 8(3), 254–272.

Linderoth, J. (2013). Beyond the Digital Divide : An Ecological Approach to Game- Play. Transactions of the Digital Game Research Association, 1(1), 1–17.

Livingstone, S., Marsh, J., Plowman, L., Ottovordemgentschenfelde, S., & Fletcher-Watson, B. (2015). Young children(0-8) and digital technology.

Löwgren, J., & Stolterman, E. (2004). Thoughtful interaction design: A design perspective on information technology. Mit Press.

Macaranas, A. (2013). The Effects of Intuitive Interaction Mappings on the Usability of Body-based Interfaces. SIMON FRASER UNIVERSITY. Retrieved from http://summit.sfu.ca/item/12635

Macaranas, A., Antle, A., & Riecke, B. E. (2015). What is Intuitive Interaction? Balancing Users’ Performance and Satisfaction with Natural User Interfaces. Interacting with Computers, iwv003.

Maguire, M. (2001). Context of use within usability activities. International Journal of Human-Computer Studies, 55(4), 453–483.

Mahmoud, M., & Robinson, P. (2011). Interpreting hand-over-face gestures. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6975 LNCS(PART 2), 248–255. http://doi.org/10.1007/978-3-642-24571-8_27

Bibliography 197

Maier, J. R. A., & Fadel, G. M. (2009). Affordance based design: a relational theory for design. Research in Engineering Design, 20(1), 13–27.

Malafouris, L. (2013). How Things Shape the Mind. MIT Press.

Malinverni, L., Ackermann, E., & Pares, N. (2016). Experience as an object to think with: From sensing-in-action to making-sense of action in full-body interaction learning environments. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 332–339).

Marshall, P., Fleck, R., Harris, A., Rick, J., Hornecker, E., Rogers, Y., … Dalton, N. S. (2009). Fighting for control: children’s embodied interactions when using physical and digital representations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2149–2152).

Marshall, P., Price, S., & Rogers, Y. (2003). Conceptualising tangibles to support learning. Proceeding of the 2003 Conference on Interaction Design and Children - IDC ’03, (c), 101. http://doi.org/10.1145/953540.953551

Mattel, Sekkoïa, & Tavitian, B. (2000). Blokus.

McCall, R. B. (1974). Exploratory manipulation and play in the human infant. Monographs of the Society for Research in Child Development, 1–88.

McEwan, M., Blackler, A., Johnson, D., & Wyeth, P. (2014). Natural Mapping and Intuitive Interaction in Videogames. CHI PLAY ’14 Proceedings of the First ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play, 191–200. http://doi.org/http://dx.doi.org/10.1145/2658537.2658541

McGrenere, J., & Ho, W. (2000). Affordances: Clarifying and evolving a concept. In Graphics interface (Vol. 2000, pp. 179–186).

Meier, B. P., & Robinson, M. D. (2004). Why the sunny side is up associations between affect and vertical position. Psychological Science, 15(4), 243–247.

Mellar, H., & Bliss, J. (1994). Introduction: modelling and education.

Menary, R. (2010). The extended mind. Cambridge, MA: MIT Press.

Microsoft. (2010). XBOX Kinect.

Mihajlov, M., Law, E. L.-C., & Springett, M. (2015). Intuitive Learnability of Touch Gestures for Technology-Na{ï}ve Older Adults. Interacting with Computers, iwu044.

Mihajlov, M., Law, E. L.-C., & Springett, M. (2015). Intuitive Learnability of Touch Gestures for Technology-Naive Older Adults. Interacting with Computers, 27(3), 344–356. http://doi.org/10.1093/iwc/iwu044

Milgram, P., & Colquhoun, H. (1999). A taxonomy of real and virtual world display integration. Mixed Reality: Merging Real and Virtual Worlds, 5–30.

Minsky, M. (1988). Society of mind. Simon and Schuster.

Mistry, P., & Maes, P. (2009). SixthSense: a wearable gestural interface. In ACM SIGGRAPH ASIA 2009 Sketches (p. 11).

198 Bibliography

Moeller, J., & Kerne, A. (2012). ZeroTouch: an optical multi-touch and free-air interaction architecture. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2165–2174).

Mohan, G., Blackler, A., & Popovic, V. (2015). Using conceptual tool for intuitive interaction to design intuitive website for SME in India: A case study. In IASDR 2015: Interplay Proceedings (pp. 1500–1521). Brisbane, QLD.

Mohs, C., Hurtienne, J., Israel, J. H., Naumann, A., Kindsmüller, M. C., Meyer, H. A., & Pohlmeyer, A. (2006). IUUI--intuitive use of user interfaces. Usability Professionals, 6(130–133).

Montemayor, J., Druin, A., Farber, A., Simms, S., Churaman, W., & D’Amour, A. (2002). Physical programming: designing tools for children to create physical interactive environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 299–306). New York, NY, USA: ACM. http://doi.org/10.1145/503376.503430

Montessori, M. (2011). Dr. Montessori’s own handbook. Schocken. (Originally published in 1914)

Montessori, M. (2013). The montessori method. Transaction Publishers.

Muller-Tomfelde, C., & Fjeld, M. (2012). Tabletops: Interactive Horizontal Displays for Ubiquitous Computing. Computer, 45(2), 78–81.

Muro, C., Escobedo, R., Spector, L., & Coppinger, R. P. (2011). Wolf-pack (Canis lupus) hunting strategies emerge from simple rules in computational simulations. Behavioural Processes, 88(3), 192–197.

Myers, D. (2007). The powers and perils of intuition. Scientific American Mind, 18(3), 24–31.

Nardi, B. A. (1996). Context and consciousness: activity theory and human-computer interaction. mit Press.

Nathan, M. J. (2008). An embodied cognition perspective on symbols, gesture and grounding instruction. Symbols, Embodiment and Meaning: A Debate, 18, 375–396.

Natural Motions, & Scott, L. (2011). Jenga app.

Naumann, A., & Hurtienne, J. (2010). Benchmarks for intuitive interaction with mobile devices. In Proceedings of the 12th international conference on Human computer interaction with mobile devices and services (pp. 401–402).

Naumann, A., Hurtienne, J., Israel, J., Mohs, C., Kindsmüller, M., Meyer, H., & Hußlein, S. (2007). Intuitive Use of User Interfaces: Defining a Vague Concept. In D. Harris (Ed.), Engineering Psychology and Cognitive Ergonomics SE - 14 (Vol. 4562, pp. 128–136). Springer Berlin Heidelberg. http://doi.org/10.1007/978-3-540-73331-7_14

Naumann, A., Hurtienne, J., Kindsmüller, M. C., Pohlmeyer, A. E., Clemens, C., Knapheide, C., … Israel, J. H. (2009). Exploring Design Criteria for Intuitive Use. In Mensch \& Computer Workshopband (pp. 73–75).

Bibliography 199

Nielsen, J. (1994). Usability inspection methods. Conference Companion on Human Factors in Computing Systems - CHI ’94, 25(1), 413–414. http://doi.org/10.1145/259963.260531

Noddings, N., & Shore, P. J. (1984). Awakening the Inner Eye. Intuition in Education. ERIC.

Noë, A. (2004). Action in perception. MIT press.

Noldus. (1989). Observer XT.

Norman, D. (1993). Things that make us smart: Defending human attributes in the age of the machine. Basic Books.

Norman, D. (1999). Affordance, conventions, and design. Interactions, 6(3), 38–43. http://doi.org/10.1145/301153.301168

Norman, D. (2010). Natural user interfaces are not natural. Interactions, 17(3), 6. http://doi.org/10.1145/1744161.1744163

Norman, D. (2013). The design of everyday things: Revised and expanded edition. Basic books.

Nunnally, J., & Lemond, C. (1974). Exploratory behavior and human development. Advances in Child Development and Behavior, 8, 59–109.

O’Brien, M. (2010). Understanding human-technology interactions: The role of prior experience and age. ProQuest Dissertations and Theses. Georgia Institute of Technology.

O’hara, K., Harper, R., Mentis, H., Sellen, A., & Taylor, A. (2013). On the naturalness of touchless: Putting the “interaction” back into NUI. ACM Transactions on Computer-Human Interaction (TOCHI), 20(1), 5.

O’Malley, C. E., Draper, S. W., & Riley, M. S. (1984). Constructive interaction: A method for studying human-computer-human interaction. In Proceedings of IFIP Interact (Vol. 84, pp. 269–274).

Olson, I. C., Atrash Leong, Z., Wilensky, U., & Horn, M. S. (2011). It’s Just a Toolbar!: Using Tangibles to Help Children Manage Conflict Around a Multi-touch Tabletop. In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 29–36). New York, NY, USA: ACM. http://doi.org/10.1145/1935701.1935709

Pallarino, T., Free, A., Mutuc, K., & Yarosh, S. (2016). Feeling Distance: An Investigation of Mediated Social Touch Prototypes. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (pp. 361–364).

Payne, S. J. (1991). Display-based action at the user interface. International Journal of Man-Machine Studies, 35(3), 275–289.

Pearl, J. (1984). Heuristics: intelligent search strategies for computer problem solving. Addison-Wesley Pub. Co., Inc., Reading, MA.

200 Bibliography

Perry, M., Church, R. B., & Goldin-Meadow, S. (1988). Transitional knowledge in the acquisition of concepts. Cognitive Development, 3(4), 359–400.

Piaget, J. (1952). The Origins of Intelligence in Children. International University Press.

Piaget, J. (2013). The construction of reality in the child (Vol. 82). Routledge. (Originally published in 1955)

Plessner, H., Betsch, C., & Betsch, T. (2011). Intuition in judgment and decision making. Psychology Press.

Popovic, V. (2003). An approach to knowledge generation by research and its utilisation in practice: situating doctoral research around the artifacts.

Popovic, V., Kraal, B. J., Blackler, A. L., & Chamorro-Koc, M. (2012). Observational research and verbal protocol methods. In P. Israsena (Ed.), Design Research Society (DRS) 2012 Conference - Research : Uncertainty, Contradiction and Value (pp. 311–324). Bangkok, Thailand: Department of Industrial Design, Faculty of Architecture, Chulalongkorn University. Retrieved from http://eprints.qut.edu.au/54222/

Pure Digital Technologies, & Cisco Systems. (2006). Flip Video Cameras.

Queensland Studies Authority. (2006). Early years curriculum guidelines, 1. Retrieved from http://www.qsa.qld.edu.au/downloads/learning/ey_cg_06.pdf

Rapp, D. N., & Kurby, C. A. (2008). The “ins” and “outs” of learning: Internal representations and external visualizations. Visualization: Theory and Practice in Science Education, 29–52.

Rasmussen, J. (1985). The role of hierarchical knowledge representation in decisionmaking and system management. IEEE Transactions on Systems, Man, and Cybernetics, (2), 234–243.

Rasmussen, J. (1993). Deciding and doing: Decision making in natural contexts. Decision Making in Action: Models and Methods, 158–171.

Regenbrecht, H., Lum, T., Kohler, P., Ott, C., Wagner, M., Wilke, W., & Mueller, E. (2004). Using Augmented Virtuality for Remote Collaboration. Presence: Teleoperators and Virtual Environments, 13(3), 338–354. http://doi.org/10.1162/1054746041422334

Resnick, L. B. (1986). The development of mathematical intuition. Perspectives on Intellectual Development: The Minnesota Symposia on Child Psychology, 19, 159–194.

Resnick, M., Martin, F., Berg, R., Borovoy, R., Colella, V., Kramer, K., & Silverman, B. (1998). Digital Manipulatives : New Toys to Think With, (April), 281–287.

Ricci, A., Piunti, M., Tummolini, L., & Castelfranchi, C. (2015). The mirror world: Preparing for mixed-reality living. IEEE Pervasive Computing, 14(2), 60–63. http://doi.org/10.1109/MPRV.2015.44

Bibliography 201

Robert, D., Wistorrt, R., Gray, J., & Breazeal, C. (2011). Exploring mixed reality robot gaming. Proceedings of the Fifth, 125–128. http://doi.org/10.1145/1935701.1935726

Robertson, T. (1997). Cooperative work and lived cognition: a taxonomy of embodied actions. Proceedings of the 5th Conference on European Conference on Computer-Supported Cooperative Work, 205–220. http://doi.org/10.1007/978-94-015-7372-6_14

Robins, B., & Dautenhahn, K. (2014). Tactile interactions with a humanoid robot: novel play scenario implementations with children with autism. International Journal of Social Robotics, 6(3), 397–415.

Rogers, Y. (2011). Interaction design gone wild: striving for wild theory. Interactions, 18(4), 58–62. http://doi.org/10.1145/1978822.1978834

Rosen, R. (2012). Anticipatory systems. In Anticipatory systems (pp. 313–370). Springer.

Rosenbaum, E., Eastmond, E., & Mellis, D. (2010). Empowering programmability for tangibles. In Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction (pp. 357–360). New York, NY, USA: ACM. http://doi.org/10.1145/1709886.1709974

Rosenthal, R., & Rosnow, R. L. (1991). Essentials of behavioral research: Methods and data analysis. In Essentials of behavioral research: Methods and data analysis (p. p.361). McGraw-Hill Humanities Social.

Ruff, H., & Saltarelli, L. (1993). Exploratory play with objects: basic cognitive processes and individual differences. New Directions for Child Development, (59), 5–16. http://doi.org/10.1002/cd.23219935903

Rutter, M. (1985). Family and school influences on cognitive development. Journal of Child Psychology and Psychiatry, 26(5), 683–704.

Sakai, M., & Sugano, M. (2016). Human tracking system embedded in stuffed animal. In Global Humanitarian Technology Conference (GHTC), 2016 (pp. 842–844).

Salomon, G. (1997). Distributed cognitions: Psychological and educational considerations. Cambridge University Press.

Santos, B. S., Cardoso, J., Ferreira, B. Q., Ferreira, C., & Dias, P. (2016). Developing 3D Freehand Gesture-Based Interaction Methods for Virtual Walkthroughs: Using an Iterative Approach. In Handbook of Research on Human-Computer Interfaces, Developments, and Applications (pp. 52–72). IGI Global.

Sapounidis, T., Demetriadis, S., & Stamelos, I. (2015). Evaluating children performance with graphical and tangible robot programming tools. Personal and Ubiquitous Computing, 19(1), 225–237. http://doi.org/10.1007/s00779-014-0774-3

Saxberg, B. (1987). Projected free fall trajectories. Biological Cybernetics, 56(2), 159–175.

202 Bibliography

Schafer, G. J., Green, K. E., Walker, I. D., Lewis, E., Fullerton, S. K., Soleimani, A., … others. (2013). Designing the LIT KIT: An Interactive, Environmental Mixed-Technology Robotic System for Enhancing Children’s Picture-book Reading.

Schneider, B., Wallace, J., Blikstein, P., & Pea, R. D. (2013). Preparing for future learning with a tangible user interface: The case of neuroscience, 6(2), 117–129.

Schon, D. (1982). Intuitive thinking? A digression stimulated by U-shaped curves. U-Shaped Behavioral Growth, 227–247.

Schore, A. (2010). The right brain implicit self: A central mechanism of the psychotherapy change process. Knowing, Not-Knowing and Sort of Knowing: Psychoanalysis and the Experience of Uncertainty, 177–202.

Scott, L. (2006). Jenga. Board game], Hasbro: Pawtucket, RI.

Searle, J. R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3(3), 417–424.

Seo, D. W., & Lee, J. Y. (2013). Direct hand touchable interactions in augmented reality environments for natural and intuitive user experiences. Expert Systems with Applications, 40(9), 3784–3793.

Seo, J., Arita, J., Chu, S., Quek, F., & Aldriedge, S. (2015). Material Significance of Tangibles for Young Children. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 53–56). New York, NY, USA: ACM. http://doi.org/10.1145/2677199.2680583

Shaer, O., & Hornecker, E. (2010). Tangible user interfaces: past, present, and future directions. Foundations and Trends in Human-Computer Interaction, 3(1--2), 1–137.

Shaer, O., & Jacob, R. J. K. (2009). A Specification Paradigm for the Design and Implementation of Tangible User Interfaces. ACM Trans. Comput.-Hum. Interact., 16(4), 20:1--20:39. http://doi.org/10.1145/1614390.1614395

Shapiro, L. (2011). Embodied cognition. New York: Routledge.

Shapiro, L. A. (2004). The mind incarnate.

Sharlin, E., Watson, B., Kitamura, Y., Kishino, F., & Itoh, Y. (2004). On tangible user interfaces, humans and spatiality. Personal and Ubiquitous Computing, 8(5), 338–346.

Sherman, L., Druin, A., & Montemayor, J. (2001). StoryKit: tools for children to build room-sized interactive experiences. CHI’01 Extended …, 197–198. Retrieved from http://dl.acm.org/citation.cfm?id=634186

Simmons, J. P., & Nelson, L. D. (2006). Intuitive confidence: choosing between intuitive and nonintuitive alternatives. Journal of Experimental Psychology: General, 135(3), 409.

Bibliography 203

Slovic, P., Finucane, M. L., Peters, E., & MacGregor, D. G. (2007). The affect heuristic. European Journal of Operational Research, 177(3), 1333–1352. article.

Søndergaard, D. M. (2013). Virtual materiality, potentiality and subjectivity: How do we conceptualize real-virtual interaction embodied and enacted in computer gaming, imagination and night dreams&quest. Subjectivity, 6(1), 55–78.

SPSS Statistics. (2016). SPSS Statistics. Armonk, NY: IBM Corp.

Strawson, G. (1999). The self and the SESMET. Journal of Consciousness Studies, 6, 99–135.

Streeck, J. (2013). Interaction and the living body. Journal of Pragmatics, 46(1), 69–90. http://doi.org/10.1016/j.pragma.2012.10.010

Streeck, J., Goodwin, C., & LeBaron, C. (2011). Embodied interaction in the material world: An introduction. Embodied Interaction: Language and Body in the Material World, 1–26.

Swaak, J., & De Jong, T. (2001). Discovery simulations and the assessment of intuitive knowledge. Journal of Computer Assisted Learning, 17(3), 284–294. http://doi.org/10.1046/j.0266-4909.2001.00183.x

Swann, L., Popovic, V., Thompson, H., Blackler, A., & Kraal, B. (2015). Relationships between user experience and intuitiveness of visual and physical interactions. In Proceedings of the 6th IASDR (The International Association of Societies of Design Research Congress) (pp. 1900–1916).

Tangible Play. (2014a). Newton.

Tangible Play. (2014b). Osmo. Retrieved from https://www.playosmo.com/en/?gclid=Cj0KCQjwub7NBRDJARIsAP7wlT8ouMB5AovBX3Yt55E94hkb_efik1uIcXmNCIYA13YTAm4lLvHyhwwaAjP2EALw_wcB

Taraborelli, D., & Mossio, M. (2008). On the relation between the enactive and the sensorimotor approach to perception. Consciousness and Cognition, 17(4), 1343–1344.

TEI 2017. (2017). TEI 2017 call for papers. Retrieved November 24, 2014, from http://allconferencecfpalerts.com/cfp/view.php?eno=4132

Terrenghi, L., Kirk, D., Sellen, A., & Izadi, S. (2007). Affordances for manipulation of physical versus digital media on interactive surfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 1157–1166).

Thelen, E. (2008). Grounded in the world: Developmental origins of the embodied mind. Developmental Perspectives on Embodiment and Consciousness, 99–129.

Thelen, E., & Smith, L. B. (1996). Dynamic systems approach to the development of cognition and action. Cambridge, USA: The MIT Press.

Thelen, E., & Smith, L. B. (2006). Dynamic systems theories. Handbook of Child Psychology.

204 Bibliography

Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59(236), 433–460.

Turner, J. S. (2013). Super organisms and superindividuality: the emergence of individuality in a social insect assemblage. From Groups to Individuals, 219–242.

Uden, L., & Dix, A. (2000). Iconic interfaces for kids on the Internet. In IFIP world computer congress (pp. 279–286).

Ullmer, B., & Ishii, H. (2000). Emerging frameworks for tangible user interfaces. IBM Systems Journal, 39(3.4), 915–931.

Ullrich, D., & Diefenbach, S. (2010). From magical experience to effortlessness: an exploration of the components of intuitive interaction. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (pp. 801–804).

van den Hoven, E., & Mazalek, A. (2011). Grasping gestures: Gesturing with physical artifacts. AI EDAM, 25(3), 255–271.

Vanity Fair. (1917, December). Lazy Susan.

Veeriah, V., Pilarski, P. M., & Sutton, R. S. (2016). Face valuing: Training user interfaces with facial expressions and reinforcement learning. arXiv Preprint arXiv:1606.02807.

Vera, A. H., & Simon, H. A. (1993). Situated action: A symbolic interpretation. Cognitive Science, 17(1), 7–48.

Vogiatzaki, E., Gravezas, Y., & Solutions, I. S. A. T. (2013). Rehabilitation System for Stroke Patients using Mixed-Reality and Immersive User Interfaces.

Vogt, P. (2002). The physical symbol grounding problem. Cognitive Systems Research, 3(3), 429–457.

Wang, K. (2015). Research design in counseling. Nelson Education.

Want, R., Fishkin, K. P., Gujar, A., & Harrison, B. L. (1999). Bridging physical and virtual worlds with electronic tags. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 370–377).

Wensveen, Overbeeke, Djajadiningrat, & Kyffin. (2004). Freedom of fun, freedom of interaction. Interactions, 11(5), 59–61.

Whitebread, D., Basilio, M., Kuvalja, M., & Verma, M. (2012). The importance of play: A report on the value of children’s play with a series of policy recommendations. Brussels, Belgium: Toys Industries for Europe.

Wickens, C. D., Gordon, S. E., Liu, Y., & Lee, J. (1998). An introduction to human factors engineering.

Williams, A., Kabisch, E., & Dourish, P. (2005). From interaction to participation: Configuring space through embodied interaction. Proceedings of the Ubicomp 2005, LNCS 3660, 287–304. http://doi.org/10.1007/11551201_17

Bibliography 205

Wilson, A., & Golonka, S. (2013). Embodied Cognition is Not What you Think it is. Frontiers in Psychology, 4(Article 58), 1–13. http://doi.org/10.3389/fpsyg.2013.00058

Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9(4), 625–36. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12613670

Wundt, W. (1973). The language of gestures (Vol. 6). Walter de Gruyter.

Yoon, S. H., Huo, K., & Ramani, K. (2016). TMotion: Embedded 3D mobile input using magnetic sensing technique. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 21–29).

Yu, B., Bongers, N., Van Asseldonk, A., Hu, J., Funk, M., & Feijs, L. (2016). LivingSurface: Biofeedback through Shape-changing Display. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 168–175).

Zaman, B., Abeele, V., Markopoulos, P., & Marshall, P. (2009). Tangibles for children,: the challenges. {CHI EA} ’09: Proceedings of the 27th International Conference on Human Factors in Computing Systems, 4729–4732.

Zigelbaum, J., Kumpf, A., Vazquez, A., & Ishii, H. (2008). Slurp: tangibility spatiality and an eyedropper. In CHI’08 Extended Abstracts on Human Factors in Computing Systems (pp. 2565–2574).

Zsambok, C. E., & Klein, G. E. (1997). Naturalistic decision making. Lawrence Erlbaum Associates, Inc.

Zuckerman, O., Arida, S., & Resnick, M. (2005). Extending tangible interfaces for education: digital montessori-inspired manipulatives. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 859–868).

Zuckerman, O., Gal, T., Keren-Capelovitch, T., Karsovsky, T., Gal-Oz, A., & Weiss, P. L. T. (2016). DataSpoon: Overcoming Design Challenges in Tangible and Embedded Assistive Technologies. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 30–37).

Appendices 207

Appendices

Appendix A

STRENGTH OF AGREEMENT BASED ON ICC VALUES

Table 12 Strength of agreement based on ICC values, adapted from (Koo & Li, 2016)

ICC Strength of Agreement Less than 0.5 Poor Between 0.5 and 0.75 Moderate Between 0.75 and 0.9 Good Greater than 0.9 Excellent

208 Appendices

Appendix B

PARAMETRIC AND NON-PARAMETRIC STATISTICAL ANALYSIS

Table 13 Differences between assumptions for parametric and non-parametric data analysis

methods, adapted from Field (2008)

Parametric Non-parametric Assumed distribution Normal Any Assumed variance Homogeneous Any Type of data Ratio or Interval Ordinal or Nominal Data set relationships Independent Any Measures of central tendency Mean Median Benefits Draws more conclusions Simplicity, less affected by

outliers Correlation test Pearson Spearman Between subjects, 2 groups Independent measures t-test Mann-Whitney test Between subjects, greater than 2 groups

Independent measures one-way ANOVA

Kruskal-Wallis test

Within subjects, 2 groups Matched pair t-test Wilcoxon test Within subjects, greater than 2 groups

Repeated measures one-way ANOVA

Friedman’s test

Appendices 209

Appendix C

THRESHOLDS FOR EFFECT SIZE

Table 14 Thresholds for interpreting effect size, adapted from Cohen (1992, p.40) for correlation and

Rosenthal & Rosnow (1991, p.361) for Mann Whitney U test.

Statistical Test

Relevant effect size Small Medium Large Very Large

Mann Whitney U Test

/√ 0.20 0.50 0.80 1.30

Correlation ^2/√ ^2 4 0.10 0.30 0.50 0.70

210 Appendices

Appendix D

Coding heuristics for Types of Interaction

Table 15 Coding heuristics for Types of Interaction

Sub-themes Example Heuristics Type of interaction

Intuitive - Unconscious reasoning (Blackler, 2008) - Less verbalisations (Bastick, 1982, 2003; Blackler, 2008; Chalmers, 1995) - High degrees of certainty and confidence about decisions and interactions (Blackler, 2008; Hammond, 1993; Simmons & Nelson, 2006) - Fast decision making (Plessner, Betsch, & Betsch, 2011)

Non-Intuitive - Conscious reasoning (Blackler, 2008) - More Verbalisations of behaviour and interactions (Bastick, 1982, 2003; Blackler, 2008) - high degrees of uncertainty and lack of confidence about decisions and interactions (Blackler, 2008; Simmons & Nelson, 2006) - Slow decision making (Plessner, Betsch, & Betsch, 2011) - Attention to visual features in interactions (Liu & Gale, 2011)

Partially-Intuitive - Limited verbalisations (Bastick, 1982, 2003; Blackler, 2008; Chalmers, 1995) - A combination of intuitive and non-intuitive processes (some automation, but higher level processes requiring attentive control (Baylor, 2001; Levi, 2015)

Appendices 211

Appendix E

CODING HEURISTICS FOR ASPECTS OF EMBODIMENT

Table 16 Coding heuristics for Aspects of Embodiment

Theme Groups

Sub-themes Example Heuristics


Physical affordance

- Interactions and behaviours determined by physical and material properties (Gibson, 1974/2014) - Rely on natural clues to determine actions offered by interfaces (Dotov et al., 2012)

Perceived affordance

- Interactions and behaviours determined by prior experience with similar things (Blackler et al., 2010) - Learned conventions (Norman, 2013) - Rely on deliberate clues inserted by designer for specific interactions with the interface (Dotov et al., 2012)

Emergence - Adaption in behaviour and interactions as dynamic systems evolve in their properties (Maier & Fadel, 2009) - Adaption in behaviour and interactions as knowledge evolves over time (Allen & Strathern, 2003)

Scaffolding - Offloading tasks into epistemic actions (Kirsh & Maglio, 1994) - Use of the environment, physical objects, tools, processes, and support mechanisms in interactions (Loorbach, Karreman, & Steehouder, 2013).

Cooperative activity

- Well-articulated division of labour (Xiao, 2005) - interactions performed by different people to achieve a common goal (Terrenghi et al., 2007). - Integrate contributions of all people involved in the activity (D. W. Johnson & Johnson, 1994) - Access the status information (Felemban, Gardner, Callaghan, & Pena-Rios, 2017) - Visibility of others interactions and behaviours (Felemban et al., 2017) - Delegate work to the rest of the team(Frederking, Cruz, Overbeeke, & Baskinger, 2007) - Lack of intrusion upon other’s activities (Schneider, Wallace, Blikstein, & Pea, 2013)

212 Appendices

Appendix F

CONSENT FORM FOR CHILDREN AND PARENTS

CONSENT FORM FOR QUT RESEARCH PROJECT – Questionnaire, Interview and Observations–

Children’s intuitive interaction

QUT Ethics Approval Number 1300000826

RESEARCH TEAM CONTACTS

Ms Shital Harshad Desai Assoc Prof. Althea Blackler Prof. Vesna Popovic

0411 441 584 07 3138 7030 07 3138 [email protected] [email protected] [email protected]

STATEMENT OF CONSENT

By signing below, you are indicating that you:

Have read and understood the participant information document regarding this

project.

Have had any questions answered to your satisfaction.

Understand that if you have any additional questions you can contact the research

team.

Understand that you are free to withdraw at any time, without comment or penalty.

Understand that you can contact the Research Ethics Unit on 07 3138 5123 or email

if you have concerns about the ethical conduct of the

project.

Have discussed the project with your child and what is required of them if

participating.

Understand that the project will include an audio and video recording.

Agree to participate in the project.

Name

Appendices 213

Signature

Date

STATEMENT OF CHILD CONSENT

Your parent or guardian has given their permission for you to be involved in this research

project. They have talked to you about participating in a research project related to playing

with toys and apps. Please colour one of the faces below that shows how you feel about

taking part in this study.

Name

Date

Please return this sheet to the investigator

214 Appendices

Appendix G

CONSENT FORM FOR SCHOOL PRINCIPAL

PRINCIPAL CONSENT FORM FOR QUT RESEARCH PROJECT



RESEARCH TEAM CONTACTS

Ms. Shital Harshad Desai Assoc Prof. Althea Blackler Prof. Vesna Popovic

0411 441 584 07 3138 7030 07 3138 2669

[email protected] [email protected] [email protected]

STATEMENT OF CONSENT FROM THE PRINCIPAL

By signing below, you are indicating that you:

Have read and understood the information document regarding this project.

Have had any questions answered to your satisfaction.

Understand that if you have any additional questions you can contact the research

team.

Understand that you are free to withdraw at any time, without comment or penalty.

Understand that you can contact the Research Ethics Unit on 07 3138 5123 or email

if you have concerns about the ethical conduct of the

project.

Understand that the project will include an audio and video recording.

Agree to allow us to approach parents and children at your school and potentially

conduct experiments at the school.

Name

Signature

Appendices 215

Date

Please return this sheet to the investigator.

Appendix H

IMAGE RELEASE CONSENT FORMS

Image Release: Parents and Children

PLEASE RETURN THIS COMPLETED FORM TO: Shital Desai

[email protected]

A COPY WILL BE PROVIDED FOR YOUR RECORDS

If you agree to give consent regarding the use of your image in the audio and video

recording for research purposes, please read and complete the consent below.

PARENT/GUARDIAN CONSENT

I agree to the University using, reproducing and disclosing photographic or video

images of me as explained in the Image Release Information document, Participant

Information Sheet and Consent Form.

I agree that I will make no claim against QUT for any payment or fee for appearing

in the video recording and release QUT from any other claims arising out of the

University’s use of the images of me.

I understand that the anonymity afforded to me as a participant in the research

project “Children’s intuitive interaction” will be rescinded if I appear in this video.

By signing below, you are indicating that you have discussed participation in the

video recording with your child and you are the legal guardian to provide consent to

participate.

Name of

Parent/Guardian

Signature of

Parent/Guardian

216 Appendices

Date

CHILD CONSENT

Your parent or guardian has given their permission to use your image in the audio

and video recording for research purposes. They have discussed with you about your

participation in the video recording of your playtime and interview. Please colour

one of the faces below which shows how you feel about being audio and video

recorded.

Name

Date

Please return this sheet to the investigator.

Appendices 217

Appendix I

IMAGE RELEASE INFORMATION SHEET itle

Image Release: Parents of children participating in the research QUT Ethics Approval Number 1300000826

A photographic image (including a video recording) which is sufficiently clear to enable your child to be identified as an individual is personal information. Queensland University of Technology (QUT) seeks to comply with the Information Privacy Principles as set out in the Information Privacy Act 2009. QUT shall, from time to time, endorse a privacy policy (see www.mopp.qut.edu.au ) to ensure that personal information is used and disclosed only in ways which are consistent with privacy principles and will otherwise comply with QUT’s privacy obligations under statute. In general, personal information is not disclosed or published except where an individual’s consent has been obtained.

QUT is seeking your consent to use an image of you and/or your child in media releases and research publications.

Participation in this release is voluntary.

You and your child can still participate in the experiment even if you opt not to provide consent to use the images and videos in presentations and publications.

Your decision to participate or to not participate will in no way impact upon your current or future relationship with Ms Shital Desai, Assoc Prof. Althea Blackler, Prof. Vesna Popovic or with QUT.

If you or your child have any questions please ensure you have discussed them and are comfortable with the response before providing consent. You might choose to discuss participation with the following people:

The researchers: Shital Desai, Assoc. Prof Althea Blackler, Prof. Vesna Popovic

Family or friends.

What is the release about? Your child will be observed with another child who is known to him/her playing with a toy for 60-90 minutes. The observations will be audio and video recorded to determine how comfortable your child is playing with the toy. The video recording will be 60-90 minutes in length.

Why do you want to include my child? It is important to include your child in the recording as their facial expressions, reaction to situations and conditions and the way they use the toy will enable us to determine whether the toy is comfortable to play with.

What will you ask my child to do? Your child will be asked to play with a toy or an app along with another child who is known to your child. The observations will be followed with a brief interview where your child will be asked about his/her experience playing with the toy or the app. The playtime and interview will form the basis of the video content. The length of each filming session might vary; it is estimated that your and your child’s involvement would require a time commitment of between one to two hours.

218 Appendices

The interview will probe your child to determine the attributes contributing to the satisfaction or dissatisfaction of playing with the toy.

Are there any benefits for me or my child in taking part? While the filming and publication of this video is not expected to provide any direct benefits to you or your child, the video is not expected to be of detriment to your child either. The research community in general seeks to benefit from this audio and video recording as it would result in a framework for designing products for children that facilitate intuitive use. A gift voucher of 20$ will be given to you as a token of thanks.

Are there any risks for me or my child in taking part? We believe that there are no risks

beyond normal day‐to‐day living associated with your and your child’s participation in this project. If

your child becomes upset while playing, the filming will be stopped. It is not the intention of the

video to portray any emotional discomfort.

Confidentiality The faces and speech of all children will be included in the video. QUT understands that you and your children might not wish to be named in the video. As a result the names of all children will be excluded from the video. QUT will only identify your child in the video on the basis of their association with the researcher, i.e. child(ren) in research program.

Who will see the video? The video will be used by the researchers to analyse the data and in research presentations and the images will be used in research publications and presentations.

Can I change my mind? You can decide to withdraw your participation at any stage of the experiment. However, after the experiment and once the image or video has been used in publications and presentations or in data analysis, it will not be possible to withdraw.

I am interested – what should I do next? All persons appearing in this video will be required to sign the attached Consent Form, acknowledging that they have read and understood the Image Release Information Sheet, and agree to allow the use of their image and voice in the video for QUT research purposes.

If you have any questions about this video, please do not hesitate to contact:

Shital Desai 0411 441 584 [email protected]

Thank you for helping with this research project. Please keep this sheet for your information.

Appendices 219

Appendix J

PARTICIPANT INFORMATION SHEET FOR EXPERIMENT 1 AT

SCHOOL

PARTICIPANT’s PARENT INFORMATION SHEET FOR QUT RESEARCH PROJECT

– Experiment 1: Questionnaire, Observational Study, Interview –



RESEARCH TEAM

Principal Researcher:

Shital Desai, PhD student

Associate Researchers:

Associate Professor Althea Blackler and Professor Vesna Popovic

School of Design – Creative Industries Faculty – Queensland University of Technology (QUT)

DESCRIPTION OF RESEARCH

This study investigates methods to design products for children that are intuitive to use. It will focus on product features that enable children to interact with the products naturally and with ease. It investigates how physical interaction with products makes them easier and more intuitive to use for children. By looking at how children play with toys and with apps on a tablet, this research will inform us if children use products more naturally when they are able to sense, perceive and act to take a decision in regards to the use of the product. The outcome of this work will be a design framework to develop intuitive products for children.

220 Appendices

PARTICIPATION

Participation by your child in this study will involve the following:

Your child with your help will complete a 10‐minute questionnaire about the toys and apps

that they play with. The questionnaire is provided in the information pack to be filled out and

send it back with your child to the school along with the signed consent forms.

On the day of the experiment, your child will be paired with another child from his/her class

at the school. Together, they will play with a toy or an app for 30 minutes at the school. The

observations will be audio and video recorded for analysing the data after the experiments.

Children will be encouraged to talk and think aloud to describe how they are using the toy or

the app and why are they following certain strategies while playing the game.

The observations will be immediately followed up with a 20‐minute interview with the

children to determine their satisfaction levels playing with the toy or the app. Children will be

asked about the features of the toy or the app that made it intuitive and satisfying to use. The

interview will be audio and video recorded for analysis.

The toys that that have been shortlisted for the experiments are available in local Australian toy

shops and comply with the Australian toy standard AS/NZ 8124. The apps will be compatible to IOS

and Android platforms and will be downloaded from ITunes and Google Play. The toys and apps are

age appropriate as per recommendations on the toy packaging and the app store.

You are most welcome to stay through the experiments. However, we do ask that you do not

interfere with the experiment or help the children in playing with the toy or the app to ensure

validity of results. Please indicate in the questionnaire whether you would like to be present at the

experiments. You will be contacted preferably by email once the date and time for the experiments

is finalised.

It is not mandatory for you to sit through the experiments. Your child will be supervised and

cared for at all times.

EXPECTED BENEFITS

This research will lead to a better understanding of how to design products for children that facilitate

intuitive use. The research outcomes will benefit the design community in creating engaging

products for children.

RISKS

There will be no more risk to your child than day‐to‐day activities. Shital Desai, the chief investigator

has undergone first aid training and will be present at all times during the experiments.

PRIVACY AND CONFIDENTIALITY

All data collected will be only used for the research project titled, “Children’s intuitive interaction”. The results will be published in research publications and presented at conferences. The video recordings will be used for analysis and images will be used in research publications and presentations. However, we will not identify you or your child by names. Wherever possible, the faces in the images will be blurred. Images cannot be blurred if it is being used to describe facial

Appendices 221

expressions of your child while playing with the toy. Name and contact details of you and your child will be stored for future communications in relation to the experiments. Consent will be obtained from you and your child to use the data collected during experiments and the images of your child in publications, research presentations and any other form of research dissemination. You will be asked to sign the consent forms and the image release form. If you decide not to sign the image release form, your child's facial features in the image will be pixelated if the image is used in publications and presentations, but your child can still participate in the experiment. Research data will be stored in secured locked cabinets in the School of Design, Block D, QUT Gardens Point Campus for mandatory minimum duration of 5 years. The access to cabinets and the laboratory premises is available to authorised personnel only. The data (video recordings) will primarily be stored on a laboratory server which is located in the QUT library. The data will be imported on a QUT computer hard disk for analysis. The access to the laboratory server and the QUT computer is secured and password protected. The data will be accessed only by the researchers, Ms Shital Desai, Associate Professor Althea Blackler, Professor Vesna Popovic.

CONSENT TO PARTICIPATE

Participation in this research project is voluntary. A decision not to participate will not adversely

affect your child’s academic achievement or their relationship with their teachers or the school. You

and your child might withdraw from the study at any time without giving any reason.

We would like to ask you and your child to sign written consent forms (enclosed) to confirm your

agreement to participate.

QUESTIONS / FURTHER INFORMATION ABOUT THE PROJECT

If have any questions or require further information please contact one of the research team

members below.

Ms Shital Harshad Desai Aspro Althea Blackler Prof Vesna Popovic 0411 441 584 07 3138 7030 07 3138 2669 [email protected] [email protected] [email protected]

CONCERNS / COMPLAINTS REGARDING THE CONDUCT OF THE PROJECT

QUT is committed to research integrity and the ethical conduct of research projects. However, if you

do have any concerns or complaints about the ethical conduct of the project you might contact the

QUT Research Ethics Unit on 07 3138 5123 or email . The QUT

Research Ethics Unit is not connected with the research project and can facilitate a resolution to

your concern in an impartial manner.

222 Appendices

Thank you for helping with this research project. Please keep this sheet for your

information.

Appendix K

PARTICIPANT INFORMATION SHEET FOR EXPERIMENT 1 AT PAS

LAB

PARTICIPANT’s PARENT INFORMATION SHEET FOR QUT RESEARCH PROJECT




RESEARCH TEAM




Associate Professor Alethea Blackler and Professor Vesna Popovic


DESCRIPTION OF RESEARCH


PARTICIPATION

Appendices 223



that they play with. The questionnaire is provided in the information pack to be filled out in

your spare time and send it back along with the signed consent forms. You could also bring

them to the lab on the day of the experiment to be handed over to the researchers.

On the day of the experiment, your child will be paired with another child who is known to

him/her. Together, they will play with a toy or an app for 30 minutes at the People and

Systems lab at QUT. The observations will be audio and video recorded for analysing the data

after the experiments. Children will be encouraged to talk and think aloud to describe how

they are using the toy or the app and why are they following certain strategies while playing

the game.







and Android platforms and will be downloaded from iTunes and Google Play. The toys and apps are


You are most welcome to stay through the experiments. However, we do ask that you do not


validity of results. It is not mandatory for you to sit through the experiments. Your child will be

supervised and cared for at all times.

EXPECTED BENEFITS


intuitive use. A gift voucher of $20 will be given to you as a token of thanks. Apart from this, there

will be no direct benefit for you and your child. The research outcomes will benefit the design

community in creating engaging products for children.

RISKS




All data collected will be only used for the research project titled, “Children’s intuitive interaction”. The results will be published in research publications and presented at conferences. The video recordings will be used for analysis and images will be used in research publications and presentations. However, we will not identify you or your child by names. Wherever possible, the faces in the images will be blurred. Images cannot be blurred if it is being used to describe facial expressions of your child while playing with the toy. Name and contact details of you and your child will be stored for future communications in relation to the experiments. Consent will be obtained from you and your child to use the data collected during experiments and the images of your child in publications, research presentations and any other form of research

224 Appendices

dissemination. You will be asked to sign the consent forms and the image release form. If you decide not to sign the image release form, your child's facial features in the image will be pixelated if the image is used in publications and presentations, but your child can still participate in the experiment. Research data will be stored in secured locked cabinets in the School of Design, Block D, QUT Gardens Point Campus for mandatory minimum duration of 5 years. The access to cabinets and the laboratory premises is available to authorised personnel only. The data (video recordings) will primarily be stored on a laboratory server which is located in the QUT library. The data will be imported on a QUT computer hard disk for analysis. The access to the laboratory server and the QUT computer is secured and password protected. The data will be accessed only by the researchers, Ms Shital Desai, Associate Professor Althea Blackler, Professor Vesna Popovic.









members below.

Ms Shital Harshad Desai Assoc Prof. Althea Blackler Prof. Vesna Popovic 0411 441 584 07 3138 7030 07 3138 2669 [email protected] [email protected] [email protected]







Thank you for helping with this research project. Please keep this sheet for your information.

Appendices 225

Appendix L

PARTICIPANT INFORMATION SHEET FOR EXPERIMENT 2

PARTICIPANT’S PARENT INFORMATION SHEET FOR QUT RESEARCH PROJECT




RESEARCH TEAM




Associate Professor Althea Blackler and Professor Vesna Popovic


DESCRIPTION


PARTICIPATION


226 Appendices


that they play with. The questionnaire is provided in the information pack to be filled out in

your spare time and send it back along with the signed consent forms. You could also bring

them to the lab on the day of the experiment to be handed over to the researchers.

On the day of the experiment, your child will be paired with another child who is known to

him/her. Together, they will play with a toy for around 90 minutes, at the People and Systems

lab at QUT. The observations will be audio and video recorded for analysing the data after the

experiments. Children will be encouraged to talk and think aloud to describe how they are

using the toy or the app and why are they following certain strategies while playing the game.







and Android platforms and will be downloaded from iTunes and Google Play. The toys and apps are


You are most welcome to stay back through the experiments. However, we do ask that you do not


validity of results. It is not mandatory for you to sit through the experiments. Your child will be

supervised and cared for at all times.

EXPECTED BENEFITS


intuitive use. A gift voucher of $20 will be given to you as a token of thanks. Apart from this, there

will be no direct benefit for you and your child. The research outcomes will benefit the design

community in creating engaging products for children.

RISKS




All data collected will be only used for the research project titled, “Children’s intuitive interaction”. The results will be published in research publications and presented at conferences. The video recordings will be used for analysis and images will be used in research publications and presentations. However, we will not identify you or your child by names. Wherever possible, the faces in the images will be blurred. Images cannot be blurred if it is being used to describe facial expressions of your child while playing with the toy. Name and contact details of you and your child will be stored for future communications in relation to the experiments. Consent will be obtained from you and your child to use the data collected during experiments and the images of your child in publications, research presentations and any other form of research dissemination. You will be asked to sign the consent forms and the image release form. If you decide not to sign the image release form, your child’s facial features in the image will be pixelated if the image is used in publications and presentations, but your child can still participate in the experiment. Research data will be stored in secured locked cabinets in the School of Design, Block D, Gardens

Appendices 227

Point Campus, QUT for mandatory minimum duration of 5 years. The access to cabinets and the

laboratory premises is available to authorised personnel only. The data (video recordings) will

primarily be stored on a laboratory server which is located in the QUT library. The data will be

imported on a QUT computer hard disk for analysis. The access to the laboratory server and the QUT

computer is secured and password protected. The data will be accessed only by the researchers, Ms

Shital Desai, Associate Professor Althea Blackler, Professor Vesna Popovic.









members below.

Ms Shital Harshad Desai Assoc Prof. Althea Blackler Prof. Vesna Popovic

0411 441 584 07 3138 7030 07 3138 2669

[email protected] [email protected] [email protected]







Thank you for helping with this research project. Please keep this sheet for your

information.

228 Appendices

Appendix M

RECRUITMENT EMAIL

2)

PARTICIPATE IN RESEARCHInformation for Prospective Participants

QUT Ethics approval Number: 1300000826

The following research activity has been reviewed via QUT arrangements for the conduct of research involving human participation. If you choose to participate, you will be provided with more detailed participant information, including who you can contact if you have any concerns.

12/1/2013 Ms. Shital Desai, Assoc. Prof Althea Blackler, Prof. Vesna Popovic People and Systems Lab, Creative Industries Faculty, D Block, School of Design, Gardens Point Campus, Queensland University of Technology, Brisbane. Dear Parents, Guardians and Children,

My name is Shital Desai from the School of Design, Creative Industries Faculty, Queensland University of Technology and I’m doing a PhD titled ‘Embodied Children’s intuitive interaction’. This study investigates methods to design products for children that are intuitive to use. Intuitive use of products results in engaging, easy to use, stress free interaction with the product. The study will observe children interacting with products that are commonly used by them in everyday life such as toys and games. The toys will be appropriate to the child’s age group, example of such toys are Elefun, Tip It Balancing, Hungry Hungry Hippoes, etc. The apps are intangible versions of the toys. I’m looking for boys and girls aged 5 years to 14 years to participate in an experiment that would last for approximately 60 minutes. The experiment requires that the participant should not have played with the toy/app before the experiment. I will confirm with you about this when I contact you to schedule and organise the experiment and also on the day of the experiment. If you have played with the toy before, I will have to unfortunately decline your participation. There will be no more risk to the participants than day‐to‐day activities. The children will be under constant supervision of the researchers and their parents/guardians at all times. The chief investigator has undergone first aid training and will be present at all times during the experiments. A comprehensive Research Safety assessment has been undertaken and approved by the Health and

Appendices 229

Safety department at Creative Industries faculty, QUT. An ethics approval has been obtained from the Human Research Ethics Committee (HREC) at QUT. The project is funded by the Department of Industry, Innovation, Science, Research and Tertiary Education. The funding body will not have access to personally identifying information about you that might be obtained during the project. A gift voucher of A$20 will be given to every participant family as a token of thanks. If you would like to participate in this study, please contact the research team at [email protected] or 0411 441 584. You will be provided with further information to ensure that your decision and consent to participate is fully informed. Many thanks for your consideration of this request. Yours Sincerely, Ms. Shital Desai, Assoc. Prof Althea Blackler, Prof. Vesna Popovic

Queensland University of Technology

[email protected], 0411 441 584

[email protected] , 0410 736 494

[email protected] , 0439715407

Appendix N

EDUCATION QUEENSLAND APPROVAL

230 Appendices

Appendices 231

232 Appendices

Appendix O

QUT HUMAN RESEARCH ETHICS COMMITTEE APPROVAL

Appendices 233

234 Appendices

Appendix P

Arrangements in Monkey Blocks given to children in Experiment 2

Figure 31 Arrangements for Monkey Blocks game in black and white and colour

EMBODIED INTUITIVE INTERACTION IN CHILDREN · products with embedded electronics (referred to as...

Documents

Transcript of EMBODIED INTUITIVE INTERACTION IN CHILDREN · products with embedded electronics (referred to as...