Qualitative research and multimodality

Inna Kotchetkova

Emma Renold

Bambo Soyinka

Chris Taylor

Bella Dicks

QUALITI, School of Social Sciences

Cardiff University

Presentation 1

Multimodality

Inna Kotchetkova

Multimodality•We deal with multimodal texts on a daily basis – every time we read a newspaper, watch television, play a video or computer game, or even read a book.

•All practices are always multimodal. A theory has to be developed in which that fact is central, and a methodology producedfor forms of description in which all modes are described anddescribable together.

The roots

Since early 1970s semiotics had ways of talking about gestural, bodily, animal and other nonlinguistic ways of meaning making

(Eco, Sebeok, Barthes, 1977; 1974, 1977)

multimodality

– people communicate through:speech, accent, gesture, facial expression, gaze-direction, shape, weight, size, body posture, position & movement, dress (colour, shape, texture, etc.), hairstyle, etc.

– environments/objects communicate through:colour, texture, shape, position, opaqueness/light, weight, movement, sound, etc.

modes and media

• Modes - abstract, non-material resources of meaning-making (e.g. the ‘languages’ of image,

sound, gesture, colour, texture, narrative)

• Media - specific material forms in which modes are realised (e.g. a photograph, a computer screen, an audio recording, paint, a wall, clothing, a chisel, a poem, a block of wood)

Kress, G.and Van Leeuwen, T. (2001). Multi-modal Discourse. London: Arnold.

Communicative potential of modes

Colour is also used to convey ‘interpersonal’ meaning. Just as language allows us to realize speech acts, so colour allows us to realize ‘colour acts’. It can be and is used to do things to or for each other, e.g. to impress or intimidate through ‘power dressing’, to warn against obstructions and other hazards by painting them orange, or even to subdue people

Kress and Leuween (2002) Colour as a semiotic mode: notes for a grammar of colour. Visual Communications. Vol. 1

(3), pp. 343-368

Many modes through one media: Meaning making in talk

In talk, we mobilize language as sounded speech, and we further ‘mean’ through gestures, posture, facial expression, and other embodied resources such as physical distance, stance, movement or stasis.

Iedema, R. (2003)Multimodality, resemiotization:

extending the analysis of discourse as multi-semiotic practice. Visual Communications, Vol 2 (1).

recordings as multimodal

• voice recordings deploy:accent expression tone pitch

language volume cadence rhythm

• still images deploy:colour shape light directionality

perspective position framing size

moves from the idea that one medium communicates in a

particular way, to the idea that any one medium may employ a number

of different modes of meaning

Multimodality

from visual methods to multimodal methods

‘common semiotic principles operate in and across different modes’

‘modes can be realised in more than one medium’

Kress, G. and Van Leeuwen, T. (2001) Multimodal discourse. Arnold

Why is multimodality potentially of interest to social researchers?

A quote

observing life in educational institutions is my favourite kind of data collection. A pile of documents, a well-written book or a co-operative interviewee may excite others, but I would always prefer to sit and watch something. The data gathered by watching and listening over weeks, even months, are

for me the sweetest jams and the most aromatic oils and spices.

Sara Delamont (2001: 130)

When we deal with multimodality

1. observations of the field 2. recording of observations 3. analysis of data records4. representation of research findings

N.B. are data recorded, represented, reproduced, reduced, transformed?

Mixing media

Mixing media increasingly common in research projects

– how do the insights afforded by these different kinds of media differ (in meaningfulness, quality, validity, etc.)

– How can the researcher take account of these differences?

Questions

When different media conceived in this way are combined together in a single research project, how does the researcher take account of these discrete kinds of communicational value?

How to integrate analytic strategies for written records with image-records, for example?

Are we talking about the same kind of informational or communicational value? If not, how might these different kinds of value be reconciled?

Hypermedia ethnography projectProduction and reception of science with/in an interactive science discovery centre

How the centre creates environments and spaces for learning

How children engage with and receive these environments and spaces

The reproduction and performance of science through exhibits and theatre

Hypermedia ethnography team: Bruce Mason, Bambo Soyinka, Amanda Coffey, Matthew Williams and Emma Renold

multimedia data-recording

Written texts, visual and audio data, documentary dataFieldnotesInterview transcriptsVideo recordings: montages / raw footageAudio recordings and soundscapesStill photographyDocuments and graphics

Multimodality in the field

today’s semiotic landscape increasingly complex

designed environments ubiquitous

media increasingly mixed

‘all the modes employed contribute to meaning…’

science centre exhibits hall

communicative practice involves…

• Discourse: which discourses are in play in the environment (e.g. childhood as simple & fun; education as entertainment; science as individualised, discrete operations….)

• Design: what semiotic resources have been selected as most able to articulate the discourses in play (e.g. objects rather than words; interactive not immobile objects.)?

continued…

• Production: the selection of media, or material resources: material matter adds a level of extra meaning (e.g. photos v. paintings; solid, opaque objects rather than transparent gadgets)

• Distribution: technological means of communicating e.g. across distances

• Interpretation: how meaning is re-made by people interacting with environments

framing: a multimodal principle

• framing works across diverse media: e.g. in interior design; in sound compositions, in paintings.

• introduces breaks and discontinuities; contrasts and continuities; uses framelines and empty space

framing children’s behaviour

  

  

   

Questions to consider

• How can multimodal data help us understand the interaction between people and environment in ethnography?

• How should we capture and represent multimodality in/of the learning environment?

• Which medium should we use to record our observations? What do different modes of data representations afford?

• When different media are combined together in a single research project, how does the researcher take account of the discrete kinds of communicational value?

• How to integrate analytic strategies for written records with image-records, for example? Are we talking about the same kind of informational or communicational value? If not, how might these different kinds of value be reconciled?

Useful references• Delamont, S (2001) Fieldwork in Educational Settings. Methods, Pitfalls and

Perspectives. London and NY: Routledge.• Wolcott H (1981) Confessions of a ‘trained’ observer, in T S Popkewitz and B R

Tabachnick (eds) The Study of Schooling. New York: Praeger pp. 247-63• Kress, Gunter and Theo van Leeuwen (2001) Multimodal Discourse: The Modes and

Media of Contemporary Communication. Hodder and Stoughton Publication• Kress, G. and Van Leeuwen, T. (eds) (2001) Multimodality. London: Sage.• Kress, G. and Ogborn, Jon (1998) ‘Modes of Representation and Local

Epistemologies: The Presentation of Science in Education’, Subjectivity in the School Curriculum (Working Paper 2, Institute of Education). London: University of London.

• Iedema, R. (2003)Multimodality, resemiotization: extending the analysis of discourse as multi-semiotic practice. Visual Communications, Vol 2 (1).

• Dicks, B. Mason, B., Coffey, A. and Atkinson, P. (2006) Qualitative Research and Hypermedia: Ethnography for the Digital Age. London: sage.

The tasks

Task 1: observation in the library

• Observation: what and how to watch?• Observation by broad sweep• Observation for nothing in particular• Searching for paradoxes• Searching for the problem(s) facing the group*

Looking at and recording everything is impossible. But:1) It allows to make reflexive choices on what to focus2) Tells you what kind of observer you are3) Data gathered at this stage will inform the written

account of the setting for a reader who is unfamiliar with it

*Walcott (1981)

Task 2: recording of observations

• The ‘affordances’ of media

What modes do various media allow us to record, represent, construct?

What is the use of combining various media in our research?

• photographs v. writing (field notes)

• writing v. video

• video v. photographs

Presentation 2

Ethics and consent in the field

Emma Renold

Ethics and consent in the field

• Public Space: consent forms– Filming ‘strangers’– Visitor interviews

• School visit and fieldtrip– Ethics and research with children– Guidance and the law– Opting in, Opting out: negotiating consent with

young children and ‘adult-gatekeepers’

Information poster for staff at research site

Warning poster

• Used alongside an informational flyer.

• To let people know that they may be recorded as part of a research project.

• Used when there is no intent to identify potential subjects caught on camera.

Consent form to be used alongside an informational flyer (for images only)

Detailed Consent form used during interviews

Alderson, P. and Morrow, V. (2004) Ethics, social research and consulting with children and young people. Barnardo’s.

• Farrell, A. (2005) (eds) Ethical Research with Children, Open University Press.

Ethics and research with children

Informed consent: guidance and the law

• Legal grey areas (Gillick ruling and ‘competence’ - prioritizing children’s consent)

• Professional guidelines: children’s consent must be sought in addition to parental consent (BSA, BERA,ESRC)

• ‘Informed consent’ as always negotiated and in a state of renewal (Thorne 1987)

In the case of participatory social sciences research, consent to participate is seen as an ongoing and open-ended process. Consent here is not simply resolved through the formal signing of a consent document at the start of research. Instead it is continually open to revision and questioning. Highly formalised or bureaucratic ways of securing consent should be avoided in favour of fostering relationships in which ongoing ethical regard for participants is to be sustained, even after the study itself has been completed (ESRC, Research Ethics Framework 2006:24, para 3.2.2).

ESRC, Research Ethics Framework

Opting in, opting out: negotiating consent with adult gate-keepers• Negotiating access with teachers and parents to

achieve starting point of seeking ‘provisional’ (Flewitt 2005) consent with children

• Why we took an ‘opt-out’ approach to parental consent

• Fears and anxieties: separating out participation from dissemination (overly cautious?)

‘Yes, not sure, no, yes’: negotiating consent with young children

• Temporality of consent– Before: reluctant and keen, impossibility of

consent as one-off activity before ‘research’ begins

– During: consent ‘on the move’, actively and constantly negotiated (foregrounded with camcorder)

– After??: problematics of representation• on-going dialogue and debate

Contact details

• Inna Kotchetkova – kotchetkovai@cf.ac.uk

• Emma Renold – renold@cf.ac.uk

• Bambo Soyinka - soyinkab@cf.ac.uk

QUALITI

Cardiff University

School of Social Sciences