TH UNISERVE SCIENCE ANNUAL CONFERENCElaplab.ucsd.edu/articles/Holcombe_Pashler_2010.pdf ·...

PROCEEDINGS OF THE 16TH UNISERVE SCIENCE ANNUAL CONFERENCE SEPTEMBER 29TH- OCTOBER 1ST 2010 THE UNIVERSITY OF SYDNEY CREATING ACTIVE MINDS IN OUR SCIENCE AND MATHEMATICS STUDENTS Published by: UniServe Science, Carslaw Building (F07), The University of Sydney, NSW 2006 http://sydney.edu.au/science/uniserve_science/ ISBN: 978-0-9808597-0-6 © 2010

Proceedings of the 16th UniServe Science Annual Conference, 2010

ii

PREFACE Dr Alexandra Hugman has changed careers and will not join us in 2010. We wish her the best in the new direction she is pursuing. We continue her vision of including science and mathematics teacher educators in the Uniserve Science Conference. We look forward to the continued presence of Associate Professors Mary Peat and Ian Johnston. With this year’s Proceedings we begin a new tradition: that of Open Source publishing. Furthermore, the submission, reviewing and publication processes have all been handled by an online system. We would like to thank all involved for their patience as we have worked through teething problems. The system allows incredible benefits in terms of tracking and we look forward to a smoother flow in 2011. A series of papers from the 2009 UniServe Science Conference have been published in the International Journal of Innovation in Science and Mathematics Education (IJISME) http://escholarship.usyd.edu.au/journals/index.php/CAL. We invite authors and all attendees to seriously consider publishing their work with IJISME. This Proceeding owes its existence to the Editorial and Review Panel listed below who volunteer their own time and expertise to help improve the quality of the publication.

EDITORIAL AND REVIEW PANEL Professor Simon Bates The University of Edinburgh Ms Karen Burke da Silva Flinders University Dr Carmel Coady University of Western Sydney Associate Professor Jacquelyn Cranney The University of NSW Associate Professor Gareth Denyer The University of Sydney Dr Greg Dicinoski University of Tasmania Dr Paul Francis The Australian National University Dr Sue Gordon The University of Sydney Associate Professor Thomas Hubble The University of Sydney Dr Alexandra Hugman Northern Beaches Christian School Professor Michael Jacobson The University of Sydney Associate Professor Ian D. Johnston The University of Sydney Professor Susan M. Jones University of Tasmania Dr Paula M. Myatt The University of Queensland Dr Glennys A. O’Brien University of Wollongong Dr Steve Provost Southern Cross University Professor John W. Rice The University of Sydney Mr Ian Sefton The University of Sydney Dr Margaret Wegener The University of Queensland Dr Anna Wilson The Australian National University Dr Theresa Winchester-Seeto Macquarie University The Proceedings of the 16th UniServe Science Conference contains three types of papers.

Full Refereed Papers which have been peer reviewed by two independent experts and satisfy the Australian DEST E1 category.

Full Written Papers (non-refereed) which have been subject to editorial assessment and satisfy the Australian DEST E2 category.

Abstracts (extract of paper) which have been subject to editorial assessment and satisfy the Australian DEST E3 category.

We look forward to seeing you at the 16th UniServe Science Conference. Editor-in-Chief Associate Professor Manjula Sharma


iii

TABLE OF CONTENTS

KEYNOTE PRESENTATIONS (REFEREED)

REFEREED PAPERS

NON-REFEREED PAPERS

“Data Dumping, After the Test you Forget it all”: Seeking Deep Approaches to Science Learning with Slowmation (Student-generated Animations) Garry Hoban 2The Advancing Science by Enhancing Learning in the Laboratory (ASELL) Project: The Next Chapter Simon M. Pyke, Alexandra Yeung, Manjula D. Sharma, Simon C. Barrie, Mark A. Buntine, Karen Burke da Silva, Scott H. Kable, Kieran F Lim 7

A Comparison of the Effectiveness of an Interactive, Online Module Versus a Laboratory Based Exercise which Introduces Microscopy to First Year Biology Students. Fiona Bird 13Collaborative Laboratory for Quantitative Data Analysis Adam J.Bridgeman, Siegbert Schmid 18Training Ethical Scientists: Student Views on the Benefits of Using Animals in Learning Ashley Edwards , Susan M. Jones 24Aligning Learning and Assessment Through Adaptive Strategies in Tutorials in Physics at the University of Auckland Graham F. Foster 29Teachers’ Reflections on the Challenges of Teaching Mathematics Bridging Courses Sue Gordon, Jackie Nicholas 35Online Evidence Charts to Help Students Systematically Evaluate Theories and Evidence Alex O. Holcombe, Hal Pashler 41Scenario-based MUVE for Science Inquiry Michael J. Jacobson, Debbie Richards, Shannon Kennedy-Clark, Katherine Thompson, Charlotte Taylor, Chun Hu, Meredith Taylor, Iwan Kartiko 47Promoting Reflective Dialogue through Group Analysis of Student Feedback Lorna E. Jarrett, Tony Koppi, Damien Field 53Does a Conference act as a Catalyst for Further Publications and Collaborations? A Pilot Study of a Small Science and Mathematics Education Conference. Hazel Jones, Alexandra Hugman 60Forming Groups to Foster Collaborative Learning in Large Enrolment Courses Gwendoline A.Lawrie, Kelly E. Matthews, Lawrence R. Gahan 66Teaching inductively: games in the tertiary classroom David J. Low 72Feedback in the Sciences: What is Wanted and what is Given Meloni M. Muir, Lorraine M. Ryan, Helen Drury 79Mapping Science Subjects: A Ground Up Approach Glennys O'Brien, Lorna Jarrett, Emily Purser,Christine Brown 85Does Marcel Marceau Have a Place in the Chemistry Laboratory? Karma L. Pearce 92Measurement Uncertainty as a Threshold Concept in Physics Anna Wilson, Gerlese Åkerlind, Paul Francis, Les Kirkup, Jo McKenzie, Darren Pearce, Manjula D. Sharma 98

Can Creating Podcasts be a Useful Assignment in a Large Undergraduate Chemistry Class? Emma Bartle, Nancy Longnecker, Mark Pegrum 104Undergraduate Research and Inquiry Across a Zoology Curriculum: an Evaluation Through the Lens of External Peer Review Susan M. Jones, Paula Myatt 108Blogging Biology & Podcasting Physics: Authentic Learning via Student Creation of New Media Will Rifkin, Nancy Longnecker, Joan Leach, Lloyd Davis 113


iv

ABSTRACTS

Reflections on Teaching Computational Physics and Applied Mathematics Paul C. Abbott 116ALIUS: Active Learning in University Science - Leading Change in Australian Science Teaching Dan R. Bedgood, Mauro Mocerino, Mark Buntine, Daniel Southam, Marjan Zadnik, Simon Pyke, Kieran Lim, Gayle Morris, Brian Yates, Michael Gardiner, Adam Bridgeman 117 Physics Practicals for Engineering Students? Ragbir Bhathal 118Do our Students Really Know What we Think They Know....and What can we do About it? James Botten 119Developing Scientific Literacy for all University Graduates Karen Burke da Silva 120Teaching and Flexible Learning in Science and Entrepreneurship Courses Using Mobile Devices Peter Cottrell, Rachel Wright, Lou Reinisch 121Building a Successful Outreach Program Phil Dooley 122An Intercultural Exploration of Conceptions in Thermal Physics Helen Georgiou, Manjula D. Sharma 123Teaching Instrumental Science Globally Using a Collaborative Electronic Laboratory Notebook Brynn Hibbert, Jeremy G. Frey, Rosanne Quinnell, Mauro Mocerino, Matthew Todd, Piyapong Niamsup, Adrian Plummer , Andrew Milsted 124Question Guided Instruction: a New Tool to Improve Thai Students’ Thinking Skills in the Physics Experiment Class Jiradawan Huntula, Ratchapak Chitaree 125Why do Students Still Bother to Come to Lectures When Everything is Online? Jill Johnston, Dale Hancock, Vanessa Gysbers, Gareth Denyer 126Making Science Relevant: a Faculty-wide Initiative Towards Enhancing the Student Experience Through Authentic Learning Activities Susan M. Jones, Ruth Casper, Julian Dermoudy, Jon Osborn, Brian Yates 127Threshold Learning Outcomes for Science Graduates: a Progress Report on the Learning and Teaching Academic Standards Project Susan M Jones & Brian Yates 128Mapping the Knowledge Structure of Physics Christine Lindstrøm, Manjula D. Sharma 129Multimedia-based Link Maps – a Preliminary Report Nigel Kuan, Manjula D. Sharma 130Using Creative Assessment Tasks to Engage Students in Learning and Conceptualising the Challenging Content of Biochemistry. Simon J. Myers 131Epistemological Beliefs, Personal Characteristics, and Health-related Behaviours of Students in Health and Human Sciences Steve Provost, Stephen Myers, Airdre Grant 132Integrated Science: An Inquiry Based Interdisciplinary Science Learning Experience Pauline Ross 133Threshold Concepts: Challenging the Way We Think, Teach and Learn in Biology and Science Pauline M. Ross, Charlotte Taylor, Chris Hughes, Noel Whitaker, Louise Lutze-Mann, Vicky Tzioumis 134Supporting Student Learning and Retention in Physics, Chemistry, Mathematics and Computing – An Evaluation of Curtin University's Science Clinics Program Elisabeth Settelmaier, Marjan Zadnik 135Applying the ASELL Framework for Improvement of a First Year Physics Laboratory Program Salim Siddiqui, Daniel Southam, Mauro Mocerino, Mark Buntine, Jo Ward, Marjan Zadnik 136An Investigation into Scientific Literacy amongst University Students Michael West, Manjula D. Sharma, Ian Johnston 137


v


2

“DATA DUMPING, AFTER THE TEST YOU FORGET IT ALL”: SEEKING DEEP APPROACHES TO SCIENCE LEARNING WITH SLOWMATION (STUDENT-GENERATED ANIMATIONS) Garry Hoban ([email protected]) Faculty of Education, University of Wollongong, Wollongong NSW 2522, Australia

KEYWORDS: deep approaches, surface approaches, slowmation, animation

ABSTRACT It is not uncommon for university students to rote learn facts and formulae to memorise information for a test. Unfortunately, these surface approaches to learning are encouraged by the complex teaching and learning system embedded in the context of university courses. Where possible, academics should encourage students to develop a deep approach to learning in their subjects. “Slowmation” (abbreviated from Slow Animation) is an innovative teaching strategy that encourages students to design and make their own narrated digital animation that is played slowly at 2 frames/second to explain a concept. It is a simplified way of making animations that has been developed over the last four years and is one way for students to engage deeply with science content. This strategy encourages such an approach because students design a sequence of five multimodal representations which involves them thinking about a concept in many different ways. These digital animations explaining science concepts can be shared and critiqued by other students or instructors. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 2-6, ISBN Number 978-0-9808597-0-6

INTRODUCTION The science education literature has long been awash with concerns about the nature of learning of science in school and at university (Committee for the Review of Teaching and Teacher Education, 2003; Tytler, 2008). Despite some notable exemplars to the contrary, there is a persistent view that the teaching of science is more often about the delivery of content to students as propositional knowledge rather than encouraging deep conceptual learning by them (Davis, Petish, & Smithey, 2006; Goodrum, Hackling, & Rennie, 2001). Key to the type of learning in science courses is how and why students engage with content knowledge. Engagement in science learning is about the ways in which instructors are able to shape their practice in order to encourage students to take an interest in processing information, transforming their understanding and developing richer links between science concepts and their everyday experiences of the real world (Loughran, 2010). There are, however, many influences that a university science lecturer needs to take into consideration that shape his/her practice. These influences are caused by the complex teaching and learning system impinging on the design of a university subject. For example, many science subjects have a large amount of content to cover which is strongly influenced by the knowledge requirements determined by the subsequent subjects. Hence the type of teaching and learning context in a university subject can be viewed as a “system” (Biggs, 2003) which is influenced by the teaching content, the type of student, the type of activities and the intended outcomes. This complex relationship is shown in Figure 1.

Hoban, Seeking Deep Approaches to Science Learning with Slowmation

3

Figure 1: Biggs’ 3Ps model representing the context of university teaching system A body of literature has identified that the way university students engage with content is related to the nature of the task which can lead to students taking a “surface” or “deep” approach to learning (Marton & Saljo, 1976, 1984). According to Biggs (2003), a “surface approach arises from an intention to get the task out of the way with minimum trouble, while appearing to meet requirements. Low cognitive levels are used . . . . examples include rote learning selected content instead of understanding it, padding an essay, and listing points instead of understanding it” (p. 14). In contrast, a “deep approach arises from a felt need to engage the task appropriately and meaningfully . . . .when students feel this need-to-know, they try to focus on underlying meaning: on main ideas, themes, principles or successful applications” (p. 16). An excerpt from an interview with a first year university science student in 2010 indicates that he used a surface approach to learning as he rote learned information for exams. In particular he used the phase of “data dumping” meaning that he forgot the information immediately after the exam because he was just rote learning it without any intention to develop deep meaning (“Int” means Interviewer, “St” means Student): Int: Could you tell me a little bit about the subject that you did last semester? St: It was Bio, which is a first year university subject, obviously a biology subject, mostly about the knowledge and science of organisms, how they are made and created and stuff. Int: OK, was there much science to learn in the subject? St: Yes, there was a fair bit of like rote learning science, especially about photosynthesis, and mitosis and meiosis, about their different cell structures and stuff. A lot of it was information that you weren’t familiar with and you just had to memorise it. Int: So how did you learn it then, how did you memorise it? St: Mostly just by writing out notes and process the different steps of like photosynthesis, writing them out in order and reading them over and over. Int: It’s now three months since you’ve done the subject, did you understand most of it? St: I got the general gist of most of it, most of the time, yeah, again like I said it was just rote learning information. Int: So if you had to sit the test today how do you think you would go? St: I would probably fail, just cause its been three months from now and I since them I have data dumped it, you kind of study up before the test and then straight after, you forget it all because you never use it again. Int: OK, so how do you think your learning could be improved? St: I don’t know, I am sure there are other different strategies that I could use to probably study with so that are more in your long term memory rather than just in my short term memory just right before a test, I am not sure what those other study techniques would be that would help me. Int: So it was really just a lot of rote learning and memorisation.


4

St: Yep. Int: And what did you call it, “data dumping”? St: Yes, data dumping, Int: Data dumping St: After the test you just forget it all. Int: Oh, OK, but what if you needed the information again in another subject? St: Then I would learn it all again for it. Int: Oh right, so you learn it one by one do you? St: Yes. Int: OK, thanks It appears from the interview transcript that the type of tasks required by the students promoted a surface approach to learning relying on memorizing information to pass an exam. The challenge, therefore, for science educators is to use teaching strategies that promote deep approaches to learning to engage students in thinking about content knowledge in different ways. This means designing tasks that encourage students to interpret content and if possible re-represent it (Ainsworth, 1999; Prain & Waldrip, 2006; Tyler & Prain, In Press; Waldrip, Prain, & Carolan, 2006). One way to promote engagement by students in to offer opportunities for them to create their own digital media about science concepts. Twenty years ago, getting students to make a mini movie about a science concept was unheard of because of the expense of acquiring a movie camera and a video player. Also, digital still cameras for personal use were science fiction. But times have changed. Nearly all university students now have access to digital cameras (still or movie cameras), iPods� for playing and recording sound tracks, and computers preloaded with free movie making software. It is therefore not surprising that the most popular web sites in the world, Facebook, Wikipedia, MySpace and YouTube, are all driven by user-generated content because of this widespread accessibility to media making technology. This type of learning, using different modes of digital media is consistent with ways of learning in authentic science communities. According to Lemke (1998), “When scientists think, talk, write, work and teach, they do not just use words; they gesture and move in imaginary visual spaces defined by graphical representations and simulations. . . .they combine, interconnect, and integrate verbal text with mathematical expressions, quantitative graphs, information tables, abstract diagrams, maps, drawings, photographs and a host of unique specialised visual genres seen nowhere else” (p. 88). There is a growing acknowledgement, therefore, that university students need to use various forms of literacies — text, images, models, and voice—not only as a way of recording information, but also to facilitate learning. (Prain, 2006; Prain & Waldrip, 2006).

SLOWMATION: A SIMPLIFIED FORM OF STOP-MOTION ANIMATION A “Slowmation” (abbreviated from “slow animation”) is a stop-motion animation created by university students that played in slow motion at 2 frames/second to explain a science concept (Author, 2005, 2007, 2009). Slowmation is a simplified way of making an animation that encourages students to design a multimodal representation of their learning and integrates features of clay animation, object animation and digital storytelling. Like clay animation, slowmation uses a stop-motion technique involving the manipulation of models made out of plasticine or soft play dough as digital still photos are taken of each manual movement. Like object animation, a range of materials can be used such as plastic models, wooden, paper or cardboard cut-out models commonly found in primary classrooms to animate. Similar to digital storytelling, a key part of creating a slowmation is that a narration and authentic photos can be added by the students to explain the science concept as the models are animated. In sum, a slowmation displays the following five features: purpose — the goal of a slowmation is for students to make an animated mini-movie to explain a

science concept and through the creation process, learn about the concept. Its design can include a range of technological enhancements to improve its educational value such as narration, music, other photos, diagrams, models, labels, questions, static images, repetitions and characters.

timing — slowmations are usually played slowly at 2 frames/second, not the usual animation speed of 20-24 frames/second, needing ten times fewer photos than in clay or computer animation, hence the name “Slow Animation” or “Slowmation”;

orientation — models are made in 3D and/or 2D and usually manipulated in the horizontal plane (on the floor or on a table) and photographed by a digital still camera mounted on a tripod looking down or across at the models, which makes them easier to make, move and photograph;

Hoban, Seeking Deep Approaches to Science Learning with Slowmation

5

materials — because models do not have to stand up, many different materials can be used such as soft play dough, plasticine, 2D pictures, drawings, written text, existing 3D models, felt, cardboard cut outs and natural materials such as leaves, rocks or fruit; and

technology — students use their own digital still cameras (with photo quality set on low resolution) and free movie making software available on their computers eg IMovie or SAM Animation on a Mac or Windows Movie Maker on a PC.

In sum, slowmation greatly simplifies the process of making stop-motion animations by students manipulating 2D or 3D models often lying down on a flat surface and requiring a tenth as many photos as a normal animation because they are played ten times slower at 2 frames per second.

EXAMPLE OF UNIVERSITY STUDENTS CREATING A SLOWMATION Over the last three years, over 600 slowmations have been made by preservice teacher education students at The University of Wollongong and Monash University through a funded national research project by the Australian Research Council. The preservice teachers learn to make a slowmation for the first time in a two-hour workshop and then create one as an explanatory resource on an allocated science topic as a university assignment. This can take up to 5-10 hours and they make it at home using their own digital still camera, everyday materials and the free movie making software on their own computers. Examples have been made of many science concepts as shown in mini 1-2 minute Table 1: Five Connected Multimodal Representations in Creating a Slowmation Sequence of Representations

Action Example

Representation 1 Research — text — diagrams

University students research information about the topic on their laptops and record them by creating notes summarizing the key points.

Representation 2 Storyboarding — diagrams — text

The students design a brief storyboard called a “chunking sheet” to plan out the design of their slowmation.

Representation 3 Modelling — 3D models using playdough

The students make different models or are given existing plastic models of the science concept they are trying to represent.

Representation 4 Photographs — digital still images of the small manual movements

Students take digital still photographs of models as they are manipulated manually.

Representation 5 Animation — computer generated digital animation —narration

The students download the photos onto the computer, edit them, make static images, add a narration and export it to a QuickTime format.


6

animated movies explaining a variety of concepts such as seasons, lunar cycles, life cycles of various plants and animals, particle motion, magnets, plant reproduction, weather cycles, movement of the planets, water cycle, simple machines, mitosis, meiosis and phagocytosis. Research has shown that students develop a deep understanding of the science content when they create a slowmation because they are engaging with the content in many different ways (Hoban, 2009). In effect they are creating a sequence of five multimodal representations culminating in the animation. Table 1 summarises the sequence of five representations involved in creating a slowmation along with a photo of students making a particular representation.

DISCUSSION AND CONCLUSION Getting students to create an animation to explain a science concept has traditionally been too difficult to achieve in university classrooms either due to lack of equipment or the complexity of the process and technology. Because of its simplified technique, all students can learn how to make a slowmation in a two hour workshop and then use their own technology— a digital still camera and their own free movie making software — to design and make their own animation explaining a science concept at home. Such a university assignment encourages a deep approach to learning a science concept because they create their five multimodal representations culminating in the animation. Moreover the digital format lends itself to the students showing their animation to other students by uploading them to an internet site within the university or for public display such as to YouTube. This can result in the students sharing and evaluating each other’s animations. Note Free examples, instructions and other resources can be viewed on the project web site www.slowmation.com which was funded from Australian Research Council Discovery Grant DP O8799119.

REFERENCES Biggs, J. (2003). Teaching for quality learning at university: What the student does. Berkenshire, UK and New York: Open

University Press, McGraw-Hill. Committee for the Review of Teaching and Teacher Education. (2003). Australia's teachers: Australia's future, Advancing

innovation, science, technology and mathematics Canberra: Department of Education, Science and Training. Davis, E. A., Petish, D., & Smithey, J. (2006). Challenges new science teachers face. Review of Educational Research, 76(4),

607-651. Hoban, G. (2005). From claymation to slowmation: A teaching procedure to develop students' science understandings.

Teaching Science: Australian Science Teachers' Journal, 51(2), 26-30. Hoban, G. (2007). Using slowmation to engage preservice elementary teachers in understanding science content knowledge.

Contemporary Issues in Technology and Teacher Education, 7(2), 1-9. Hoban, G. (2009). Facilitating learner-generated animations with slowmation. In L. Lockyer, S. Bennett, S. Agostino & B. Harper

(Eds.), Handbook of Research on Learning Design and Learning Objects: Issues, Applications, and Technologies (pp. 313-330). Hershey, PA: IGI Global.

Hubber, P., Tytler, R., & Haslam, F. (2010). Teaching and learning about force with a representational focus: Pedagogy and teacher change. Research in Science Education, 40(1), 5-28.

Lambert, J. (2002). Digital storytelling: Capturing lives, creating community. Berkeley, CA: Digital Diner Express. Lemke, J. (1998). Multiplying meaning: Visual and verbal semiotics in scientific text. In J. R. Martin & R. Veel (Eds.), Reading

science: Critical and functional perspectives on discourses of science (pp. 87-113). New York: Routledge. Marton, F., & Saljo, R. (1976). On qualitative differences in learning: Outcomes and process. British Journal of Educational

Psychology, 46, 4-11. Marton, F., & Saljo, R. (1984). Approaches to learning. In F. Marton, D. Hounsell & N. J. Entwistle (Eds.), The experience of

learning. Edinburgh: Scottish Academic Press. Prain, V. (2006). Learning from writing in secondary science: Some theoretical and practical implications. International Journal

of Science Education, 28(2-3), 179-201. Prain, V., & Waldrip, B. (2006). An exploratory study of teachers' and students' use of multi-modal representations of concepts

in primary science. International Journal of Science Education, 28(15), 1843-1866. Tyler, R., & Prain, V. (In Press). A framework for re-thinking learning in science from recent cognitive perspectives. International

Journal of Science Education. Waldrip, B., Prain, V., & Carolan, J. (2006). Learning junior secondary science through multi-modal representations. Electronic

Journal of Science Education, 11(1). Waldrip, B., Prain, V., & Carolyn, J. (2010). Using mulit-modal representations to improve learning in junior secondary science.

Research in Science Education, 40(1), 65-80.

Pyke, Yeung, Kable, Sharma, Barrie, Buntine, Burke da Silva, Lim, The ASELL Project: The Next Chapter

7

THE ADVANCING SCIENCE BY ENHANCING LEARNING IN THE LABORATORY (ASELL) PROJECT: THE NEXT CHAPTER Simon M. Pykea, Alexandra Yeungb, Scott H. Kableb, Manjula D. Sharmac, Simon C. Barried, Mark A. Buntinee, Karen Burke da Silvaf, Kieran F. Limg

Presenting author Simon Pyke ([email protected]) aSchool of Chemistry and Physics, The University of Adelaide, Adelaide SA 5005, Australia bSchool of Chemistry, The University of Sydney, Sydney NSW 2006, Australia cSchool of Physics, The University of Sydney, Sydney NSW 2006, Australia dInstitute for Teaching and Learning, The University of Sydney, Sydney NSW 2006, Australia eDepartment of Chemistry, Curtin University of Technology, Perth WA 6845, Australia fSchool of Biological Sciences, Flinders University, Adelaide SA 5001, Australia gSchool of Life and Environmental Sciences, Deakin University, Burwood Vic 3125, Australia KEYWORDS: students' experience, laboratory learning, science education, science experimental workshop, communities of practice, professional development

ABSTRACT Most researchers agree that the laboratory experience ranks as a significant factor that influences students’ attitudes to their science courses. Consequently, good laboratory programs should play a major role in influencing student learning and performance. The laboratory program can be pivotal in defining a student's experience in the sciences, and if done poorly, can be a major contributing factor in causing disengagement from the subject area. The challenge remains to provide students with laboratory activities that are relevant, engaging and offer effective learning opportunities. The Advancing Science by Enhancing Learning in the Laboratory (ASELL) project has developed over the last 10 years with the aim of improving the quality of learning in undergraduate laboratories, providing a validated means of evaluating the laboratory experience of students and effective professional development for academic staff. After successful development in chemistry and trials using the developed principles in physics and biology, the project has now expanded to include those disciplines. This paper will discuss the activities of ASELL and provide a report about the first ASELL science workshop held at the University of Adelaide in April 2010. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 7-12, ISBN Number 978-0-9808597-0-6

INTRODUCTION Laboratory activities have an important and characteristic role in science curricula (Hofstein & Mamlok-Naaman, 2007). Science educators have suggested many benefits of laboratory work in terms of both knowledge and skill development (Bennett & O'Neale, 1998; Hegarty-Hazel, 1990; Hofstein & Lunetta, 1982, 2004; Moore, 2006). It is acknowledged/accepted that effective experiments do not utilise a ‘follow the recipe’ structure (Domin, 1999) where students can “go through the motions... with their mind in neutral” (Bennett & O'Neale, 1998, p. 59). Experiments need to be designed to support student autonomy whilst allowing for cognitive engagement (Skinner & Belmont, 1993). This can be achieved by having students work together collaboratively to solve problems (Shibleym & Zimmaro, 2002), incorporating inquiry-based learning activities (Green, Elliott, & Cummins, 2004), or designing open-ended investigations (Psillos & Niedderer, 2002) (noting that pure discovery activities tend to be ineffective as they lack structure (Mayer, 2004)). Such activities not only improve motivation (Paris & Turner, 1994), but students can also scaffold each other’s learning (Coe, McDougall & McKeown, 1999). Each year across 35 Australian universities, about 20,000 students undertake chemistry units (Barrie, Buntine, Jamie, & Kable, 2001a). Almost half of student time is spent on laboratory activities (Royal Australian Chemical Institute, 2005), and these figures are assumed to be similar in the domains of biology and physics. So it is important that the opportunities afforded by these learning environments are realised. A challenge facing many educators is to provide laboratory programs that are relevant, engaging, and offer effective learning outcomes within existing constraints. A further dimension of this


8

challenge lies in the demonstration of the laboratory as a unique learning environment (Rice, Thomas & O'Toole, 2009).

THE ADVANCING SCIENCE BY ENHANCING LEARNING IN THE LABORATORY (ASELL) PROJECT The Advancing Science by Enhancing Learning in the Laboratory (ASELL) project provides a multi-institutional, collaborative approach for improving the quality of undergraduate laboratories and providing effective professional development for academic staff. ASELL is the expansion of the previous Australian Physical Chemistry Enhanced Laboratory Learning (APCELL) (Barrie et al., 2001a, 2001b, 2001c) and the Advancing Chemistry by Enhancing Learning in the Laboratory (ACELL) projects (Buntine et al., 2007; Jamie et al., 2007; Read, 2006a, 2006b). A(P)CELL began in 2000 when a number of chemistry academics noticed increasingly high levels of student dissatisfaction with their undergraduate chemistry laboratory courses. It was also apparent that many of the academics who taught chemistry at the tertiary level were not familiar with educational research related to students’ experiences in the laboratory. Therefore, the project team designed professional development activities that enhance both academic and student understanding of issues affecting student experiences in the laboratory. One of the tangible outcomes of the A(P)CELL project is a database of educationally-validated undergraduate experiments on an open-access website (www.acell.org). For an experiment to be accepted onto the ACELL database, it passed through a rigorous evaluation process (see Figure 1). Submitted experiments also included student notes, demonstrator notes, technical notes, hazard/risk assessment, and the ASELL Educational template. The Educational Template provides information on the context in which the experiment is run, the educational goals which it serves, how these goals are achieved, and an analysis of student feedback data providing evidence of students’ perceptions of the experiment.

Figure 1: Schematic of the ACELL process The first stage of the ACELL process involved the third-party testing of submitted experiments at a workshop by both academics and students and the evaluation of the educational and scientific merit of the exercise. The first APCELL workshop took place in 2001 and the first ACELL workshop was held in 2006. See Table 1 for a list of past workshops. The aims of the workshop were twofold – firstly the testing serves to demonstrate that the experiment is transferrable to a new institution, by having it set-up and run away from its home laboratory. The technical notes and student notes supplied need to provide sufficient information to anyone who is unfamiliar with the experiment. Secondly, testing provides valuable feedback to submitters on the strengths and weaknesses of the experiment. At the workshop, a community of practice is also fostered where discussions of practical educational theory take place. After an experiment completed workshop testing, it was returned to its home institution where modifications could be made before further student data was collected using the A(P)CELL Student Learning Experience (ASLE) survey. The ASLE survey consists of Likert-scale and open-response items, and the student evaluation part of the Educational Template must include a summary of the Likert-scale data and a content analysis from the open-response items. The project team and the website provide guidance as to how the analysis can be completed, including examples.

Return w/s

surveys

Returnstudentdata

Recycle

PublishedACELL

workshopIn‐semester

data collectionPeerreview

Analysis &write up

Reanalyse/rewrite“Stop Point” “Stop Point”


9

Table 1: Summary of past A(P)CELL workshops Following the analysis and provided the student data meets certain criteria, the submitter would be in a position to finalise the Educational Template and write the manuscript for publication. Complete submissions are then sent for peer review by 3 referees – a student who has participated in a workshop, a staff member of a university, and a member of the project management team. Normal editorial processes are followed where the submitters can respond to referee’s comments. Acceptance of the submission leads to the inclusion of the experiment on the ACELL website. If the submission included a full manuscript, this would result in automatic acceptance for publication in either of two chemistry education journals – Chemistry Education Research and Practice or the Australian Journal of Education in Chemistry (subject to minor editing for the appropriate journal). In 2007, the ACELL project team started to explore the possibility of applying the principles and processes developed in chemistry to other science disciplines. Exploratory workshops based on the ACELL process were held for physics (late 2007) and biology (early 2008). The success of these preliminary workshops in disciplines other than chemistry resulted in the establishment of ASELL in 2009. ASELL has four distinct goals:

1. to provide for the professional development of science academics by expanding their understanding of issues surrounding learning in the laboratory environment;

2. to facilitate the development of a community of practice of laboratory educators by providing mentoring in educational theory and practice, regular workshops, and a presence at scheduled education conferences;

3. to provide a sustainable mechanism, through involvement of the Australian Council of Deans of Science, to embed this cultural change as standard institutional practice; and

4. to conduct and enable research into learning and teaching in the laboratory environment. It was expected that the core activity for achieving the first two goals would be through the experimental workshop model using the process for evaluation of laboratory activities developed in ACELL (as shown in Figure 1). Educationally-validated undergraduate experiments that meet the acceptance criteria will be published on an open-access website (www.asell.org – this will also include all previously accepted ACELL activities). Journals for publications in the areas of physics and biology education are currently being negotiated.

THE ASELL WORKSHOP – THE UNIVERSITY OF ADELAIDE, APRIL 2010 The first ASELL Workshop was held at the University of Adelaide in April 2010. At this workshop 39 experiments were submitted for evaluation in parallel sessions across the three disciplines, biology, chemistry (including 2 biochemistry experiments) and physics. Testing of these experiments was completed over a four day period by a team of 42 academics and 41 students. In addition, a special 2-day workshop was run for Deans, Associate Deans and/or their representatives (13 delegates). Although this is the second ACELL/ASELL workshop the Deans have been invited to, it is the first workshop where there has been such a great representation. Table 2a provides a summary of the delegates who represented 15 different institutions. Table 2b shows the number and some of the types of experiments tested at each workshop. Delegates were invited to the workshop as teams (1 academic and 1 student) and paid a team registration fee. The Deans of Science at each of the participating institutions agreed to provide financial support for a team from each of the three disciplines at their institution to attend the workshop. Thus, the workshop was self funded and did not rely on external funding to run, which was the case in the past.

Experimental workshop Feb, 2001 (Sydney)

Nov, 2002 (Melbourne) Feb, 2004 (Hobart) Feb, 2006 (Sydney) Jan, 2007 (Adelaide)

July, 2009 (Sydney)


10

Table 2: (a) Summary of the delegates who attended the ASELL Science Workshop and (b) Number of experiments and some of the types of activities tested at the ASELL Workshop

The workshop was organised following the procedure shown in Figure 2. Delegates were sent an invitation to submit an experiment and attend the workshop. Academic staff delegates submitted an Expression of Interest for the experiment they wanted to evaluate. After consideration of the types of experiments submitted, academics were notified whether their experiment was accepted to be evaluated at the workshop. Following the acceptance notification, academics were required to submit all the necessary documentation for the experiment. These documents included: Student Notes – containing the background information and experimental notes which are

provided to students who are undertaking the experiment in its home institution Demonstrator Notes – containing information and instructions for the supervision of students as

they do the experiment. Technical Notes – containing all information required by technical staff in order to set up an

experiment, including a list of equipment and chemicals, estimated costs, settings for instrumentation (if appropriate), safety measures that need to be taken in the laboratory, and any other information which technical staff might require.

Hazard / Risk Assessment – this addresses both chemical and physical hazards associated with the experiment, as well as describing safety precautions.

The technical notes, experiment notes and risk assessments were passed onto the technical staff and PhD students who were employed to set up the workshop. Using the notes provided the experiments for the chemistry and biology workshops were set up in the corresponding laboratories at the University of Adelaide (setup commenced about 2 weeks before the workshop). Academics that submitted physics experiments were asked to send or bring their own equipment, except for common equipment provided on a list by the host institution. Equipment for biology and chemistry activities was provided by the host institution. Not all the experimental activities were easy to set up and some experiments required assistance from other disciplines. For example, two biochemistry experiments that were run at the chemistry workshop required equipment that was provided from biology. If there were any materials that could not be provided by the host institution, the submitters were asked to either send these beforehand or bring it with them (this was kept to a minimum). Fortunately, in most cases, enough laboratory space was available for the majority of experiments to be set up the day before they were due to be run. The PhD students who set up the experiments acted as technical staff throughout the workshop.

Figure 2: The process undertaken to set up the ASELL Science Workshop held at the University of Adelaide The workshop itself had a very packed schedule. A flowchart of a typical day’s events is shown in Figure 3. Each day involved early morning discussion sessions focussing on the educational aspects of laboratory work where delegates were guided through an educational analysis of their submitted

Invitation to submit experiments sent

Acceptance of experiments for workshop

Notes for experiment submitted

Necessary notes passed onto people setting up the

workshop

Workshop experiments set up

Staff and student delegates complete

experiments

(a) Biology Chemistry Physics Total (b) Biology Chemistry Physics

Academics 12 16 14 42 Total 12 13 14

Students 12 12 14 41

Types of labs

Dissection Titration Pendulum

Deans 5 6 2 13 Botany Synthesis Radioactivity

Directors 1 4 1 6 Enzymes Analytical chem

Optics

Total 30 41 31 102 Genetics Biochemistry Oscilloscope


11

experiment (this provided scaffolding for completion of the ASELL Educational Template). Morning and afternoon laboratory sessions (each 3 hours long) were separated by a communal discipline lunch break. The Deans started participating on the second day of experimental work and completed the same activities as the other delegates. In the laboratory sessions, academic staff delegates took on the role of a student in testing the experiments, with the exception that the academic who submitted the experiment acted as the demonstrator. All delegates (academic staff and students) were assigned to work in pairs and with different people in each laboratory session, fostering networking opportunities and furthering ASELL’s community of practice aims. The pairs that were assigned consisted of student+student, academic+academic, and academic+student. The Deans were treated as academic staff delegates and were also assigned a partner. Often, delegates, especially academics and the Deans, were forced to move beyond their comfort zone by undertaking experiments outside of their area of expertise. This was important in allowing academics to experience what students feel when confronted with a new experiment in an unfamiliar environment. An important part of each day was the debrief and discussion sessions held at the conclusion of the day’s activities. Delegates were asked to critically evaluate the experiments they undertook that day in a discussion forum with the submitter. Delegates approached these sessions very seriously, with many discussions continuing over dinner. One participant commented by saying “It was good to have discussion session in the evening to allow everyone to think about the experiments and potential improvements. It also allowed me to discuss certain experiments with people who had not actually done those experiments before, which at times led to novel ideas being developed”. In the evenings, the delegates who were not grouped by discipline, enjoyed some downtime over dinner therefore allowing for cross discipline interaction. These were the key times people from different disciplines would interact with each other due to the packed workshop schedule. Although this is the first time a workshop of this nature has been run, a delegate even felt that they wanted “…more interaction across disciplines and would have like to see some of the other experiments that were run. Perhaps even a session akin to a poster session where one could view and discuss a range of experiments”.

Figure 3: Flowchart of a typical day’s events at the ASELL Science Workshop

IMPACT OF THE ASELL WORKSHOP ON THE HOST INSTITUTION Hosting the workshop raised the profile of not only ‘what makes a good experiment’ but also the similarities of these factors across what had previously been considered to be a lack of any common ground. In concert with other curriculum renewal activities currently in progress, the workshop has provided increased opportunity for development of a more holistic approach to curriculum design,

Morning Discussion Session

Debrief/Feedback SessionDinner

Laboratory Session

Lunch

Laboratory Session


12

particularly in the core Level 1 discipline areas, with a focus on improving the student experience within the laboratory programs.

CONCLUSION The ASELL Workshop held in April 2010 at the University of Adelaide was the first workshop of its kind organised by ASELL. In the past, discipline-specific workshops had been organised, in particular for chemistry. The April workshop is the first example where experiments from all three disciplines were tested at the same time, while also allowing for cross discipline interaction during free/social time. The representation of Deans at the workshop was also much greater than at any previous workshop. The April 2010 workshop marks the start of more cross discipline interaction, conversations with the Deans and discussions about laboratory activities in the future.

ACKNOWLEDGEMENTS The ASELL project would not be possible without the financial support of the Australian Learning and Teaching Council and the support of the Australian Council of Deans of Science. Collection of data for this project was approved by the University of Sydney Human Research Ethics Committee, project number 12-2005/8807.

REFERENCES Barrie, S. C., Buntine, M. A., Jamie, I. M., & Kable, S. H. (2001a). APCELL: Developing Better Ways of Teaching in the

Laboratory. Paper presented at the Research and Development into University Science Teaching and Learning Workshop. Barrie, S. C., Buntine, M. A., Jamie, I. M., & Kable, S. H. (2001b). APCELL: The Australian Physical Chemistry Enhanced

Laboratory Learning Project. Australian Journal of Chemical Education, 57, 6-12. Barrie, S. C., Buntine, M. A., Jamie, I. M., & Kable, S. H. (2001c). Physical Chemistry in the Lab. Chemistry in Australia, 68(2),

37-38. Bennett, S. W., & O'Neale, K. (1998). Skills Development and Practical Work in Chemistry. University Chemistry Education, 2,

58-62. Buntine, M. A., Read, J., R., Barrie, S. C., Bucat, R. B., Crisp, G. T., George, A. V., Jamie, I. M., & Kable, S. H. (2007).

Advancing Chemistry by Enhancing Learning in the Laboratory (ACELL): A Model for Providing Professional and Personal Development and Facilitating Improved Student Laboratory Learning Outcomes. Chemistry Education Research and Practice, 8(2), 232-254.

Coe, E. M., McDougall, A. O., & McKeown, N. B. (1999). Is Peer-Assisted Learning of Benefit to Undergraduate Chemists? University Chemistry Education, 3, 72-75.

Domin, D. S. (1999). A Review of Laboratory Instructional Styles. Journal of Chemical Education, 76, 543-547. Green, W. J., Elliott, C., & Cummins, R. H. (2004). "Prompted" Inquiry-Based Learning in the Introductory Chemistry

Laboratory. Journal of Chemical Education, 81, 239-241. Hegarty-Hazel, E. (Ed.). (1990). The Student Laboratory and the Science Curriculum. London: Routledge. Hofstein, A., & Lunetta, V. N. (1982). The Laboratory in Science Teaching: Neglected Aspects of Research. Review of

Educational Research, 52, 201-217. Hofstein, A., & Lunetta, V. N. (2004). The Laboratory in Science Education: Foundation for the 21st Century. Science

Education, 88, 28-54. Hofstein, A., & Mamlok-Naaman, R. (2007). The Laboratory in Science Education: The State of the Art. Chemistry Education

Research and Practice, 8(2), 105-107. Jamie, I. M., Read, J. R., Barrie, S. C., Bucat, R. B., Buntine, M. A., Crisp, G. T., George, A. V., & Kable, S. H. (2007). From

APCELL to ACELL and Beyond - Expanding a Multi-Institution Project for Laboratory-Based Teaching and Learning. Australian Journal of Chemical Education, 67, 7-13.

Mayer, R. E. (2004). Should There Be a Three-Strikes Rule against Pure Discovery Learning? The Case for Guided Methods of Instruction. American Psychologist, 59, 14-19.

Moore, J. W. (2006). Let's go for an A in Lab. Journal of Chemical Education, 83, 519. Paris, S. G., & Turner, J. C. (1994). Situated Motivation. In P. R. Pintrich, D. R. Brown & C. E. Weinsein (Eds.), Student

Motivation, Cognition and Learning (pp. 213-237). Hillsdale, NJ: Erlbaum. Psillos, D., & Niedderer, H. (Eds.). (2002). Teaching and Learning in the Science Laboratory. Dordrecht: Kluwer. Read, J., R. (2006a). The Australian Chemistry Enhanced Laboratory Learning Project. Chemistry in Australia, 73(1), 3-5. Read, J., R. (2006b). Achievement of an ACELL Workshop. Chemistry in Australia, 73(9), 17-20. Rice, J. W., Thomas, S. M. & O'Toole, P. (2009). Tertiary Science Education in the 21st Century (Australian Learning &

Teaching Council) Royal Australian Chemical Institute. (2005). The Future of Chemistry Study: Supply and Demand of Chemists. from

http://www.raci.org.au/national/downloads/Future%20of%20Chemistry%20Report.pdf Shibleym, I. A., & Zimmaro, D. M. (2002). The Influence of Collaborative Learning on Student Attitudes and Performance in a

Chemistry Laboratory. Journal of Chemical Education, 79, 745-748. Skinner, E. A., & Belmont, M. J. (1993). Motivation in the Classroom - Reciprocal Effects of Teacher-Behaviour and Student

Engagement across the School Year. Journal of Educational Psychology, 85, 571-581.

Bird, A Comparison of the Effectiveness of an Interactive, Online Module versus a Laboratory Based Exercise

13

A COMPARISON OF THE EFFECTIVENESS OF AN INTERACTIVE, ONLINE MODULE VERSUS A LABORATORY BASED EXERCISE WHICH INTRODUCES MICROSCOPY TO FIRST YEAR BIOLOGY STUDENTS Fiona Bird ([email protected]) Department of Zoology, La Trobe University, Melbourne Vic 3086, Australia

KEYWORDS: biology education, learning in laboratories, online modules, learning microscopy

ABSTRACT Microscopy is an essential technical skill for biology students to master because they will use it throughout their undergraduate course and potentially in their working life. The aim of this project was to compare student learning of microscopy from an interactive, online Introduction to Microscopy module with a laboratory based exercise. Effectiveness of the two methods (online versus laboratory class) was evaluated with observations of students setting up a microscope late in semester. A quiz was also administered at both the start and end of semester to quantify the learning achieved as a result of a combination of the introductory exercise and subsequent use of microscopes during the semester-long laboratory program. Overall, the online Introduction to Microscopy module achieved learning outcomes that were equivalent to or better than the laboratory program. Quiz results from both years revealed that understanding of the function of the condenser and iris diaphragm was limited and the in-class observations confirmed that students rarely adjusted the condenser or iris diaphragm when using the microscope. Feedback from students about the effectiveness of the online module was sought with an online survey. Although response rate was low, some students identified that the content, design and interactivity of the online module assisted their learning of microscopy. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, 13-17, ISBN number 978-0-9808597-0-6

INTRODUCTION Practical work is viewed as an essential component of studying the natural sciences. The 'hands on' approach has the potential to stimulate student interest in the subject matter, teach laboratory skills, enhance the learning of knowledge, give insight into the scientific method and develop scientific attitudes such as objectivity (Gorst & Lee, 2005). Saunders and Dickinson (1979) showed that biology students who attended laboratory classes learned more biology and acquired more positive attitudes to science than lecture-only students. Practical work also gives students the opportunity to learn and practice the type of activities involved in working as scientists (Meester & Maskill, 1995). Laboratory exercises such as biological dissections offer a sensory as well as an intellectual experience and students develop a sense of personal discovery which stimulates intellectual curiosity (Kinzie, Strauss & Foss, 1993). Despite these benefits, virtual laboratory experiences are becoming more common in response to increased financial pressure, larger class sizes and reduced levels of staffing (Hughes, 2000; Peat & Franklin, 2001). Ethical issues regarding the use of animals and animal tissues for teaching purposes have also been cited as reasons for offering virtual alternatives to wet laboratory exercises (Hughes, 2000; Peat & Taylor, 2004). Virtual laboratories can also create opportunities for students to enhance their learning of scientific knowledge when equivalent wet-laboratory programs are not available or affordable (Stuckey-Mickell & Stuckey-Danner, 2007). The use of information and communication technologies (ICTs) to support and facilitate learning in higher education has increased significantly in recent years (Krause & McEwen, 2009) and first year students are highly positive about the benefits of using ICTs for study-related purposes (James, Krause & Jennings, 2010). Virtual learning environments created by ICTs, such as virtual laboratory experiences, allow flexible delivery of resources to students juggling work/study/life commitments (Franklin & Peat, 1998; Harris et al., 2001; Peat, 2000). If designed well, virtual learning environments provide a broad range of opportunities for large, diverse student cohorts with differing levels of experience with the subject matter and/or different learning styles (Krause & McEwen, 2009).


14

Research has shown that student learning of scientific knowledge is equivalent in virtual laboratory exercises and wet laboratory exercises. First year biology students at the University of Glasgow were offered an alternative to a rat dissection (models and charts) and the final exam performance (conceptual learning was tested) of students who chose the alternative did not differ significantly from that of students who did the real dissection (Downie & Meadows, 1995). Completion of virtual rather than real animal dissections has also been shown to result in equivalent student learning of anatomy and function of organs (Franklin, Peat & Lewis, 2002). Students who completed online simulations rather than actual laboratory experiments performed equally well on final assessment questions relating to the concepts illustrated by the experiment (Hughes, 2000). Virtual microscope laboratories (repositories of whole mounts and histological sections of biological material) give students access to high quality, consistent images (Peat & Taylor, 2004) without students needing to master the skill of using a microscope correctly. In one study, medical students rated a virtual microscope laboratory class higher for efficiency, clear directions, clear images, navigation of slides, focus on the information needed and accessibility than a regular microscope laboratory class (Harris et al., 2001). One of the intended learning outcomes of the first year biology subject Animal Diversity, Ecology and Behavior (ADEB) is that students have developed the skill of using a microscope correctly, so providing students with only a virtual microscope repository would not achieve that learning outcome. The compound microscope is introduced early in the semester, and ADEB students have the opportunity to practice throughout the semester as they examine microscope slide material. The old model of introducing microscopy was a laboratory based exercise comprising an integrated package of print material, laboratory exercises and video sequences. This model was replaced by a new, interactive, online Introduction to Microscopy (IM) module in 2009. The new IM online module was introduced to increase efficiencies in the use of costly, laboratory space and to reinvigorate the laboratory based exercise which was becoming increasingly difficult to run in a teaching laboratory with minimal and poorly performing audio-visual equipment. The IM module was developed in conjunction with staff at the University's Flexible Teaching and Learning Team, and combines explanatory text, video footage, static images and audio recordings to explain how a microscope works, explain magnification and describe the function of the different parts of the compound microscope. The IM module also contains problem solving exercises which allow students to interactively adjust a virtual microscope to achieve the best quality image possible. A list of 7 steps for adjusting the microscope can be printed off and taken to their next practical class. ADEB students in 2009 were given access to the online module through the University's learning management system throughout the semester. Learning achieved by engagement with the online module was reinforced with subsequent use of microscopes during the semester-long laboratory program. The aim of this project was to compare student learning of the components, function and correct adjustment of a compound microscope from a new, interactive, online Introduction to Microscopy module with an old laboratory based exercise.

METHODS The comparison was made between student cohorts in the years 2008 (laboratory based exercise) and 2009 (online module). The median ENTER (university entrance) score was calculated to confirm that the cohorts of students were of similar academic potential (67.1 and 68.5 in 2008 and 2009 respectively). Effectiveness of the laboratory program versus the online module was compared with a quiz and in-class observations of students using a microscope. The quiz was administered in week 1 of semester (prior to the compound microscope being introduced) and then again in week 12 (at the end of the practical course). The quiz was administered during a practical session and participation was voluntary and anonymous. The quiz consisted of multiple choice questions assessing knowledge of the components of a microscope and how to use it (see Table 1 for questions). The number of students who answered each question correctly was compared within and between years using chi-squared analysis. For the within-year analysis, week 1 (March) data were used to calculate the expected values and week 12 (May) data were the observed values. For the between-year analysis, year 1 data (May 2008) were used to calculate the expected values and week 12 (May 2009) data were the observed values.The in-class observations of students using microscopes were done in week 10 of semester. The observations were made by demonstrators during a practical session. Observers used a checklist of questions to make the observations (see Table 2 for checklist). Again participation was voluntary (students were approached prior to the observation) and data was recorded anonymously. Feedback from students about the effectiveness and design of the online


15

module was sought with an online survey of two open-ended questions/statements: Which two or three specific aspects of this online module have contributed most to your learning of microscopy? and Please suggest two or three specific, practical changes which could improve learning in this online module. Response rate was low because the survey was voluntary and administered online after students had completed the online module. Only a few responses to the open-ended questions/statements will be reported in this paper to illustrate key points.

RESULTS A significant increase in the number of correct responses (from week 1 to week 12 of semester) was found for three out of the six questions in 2008 and five out of the six questions in 2009 (Table 1), indicating that the online module achieved greater success at instructing students on the components of the microscope and their function than the laboratory based exercise. The comparison between years revealed no significant differences in the numbers of correct responses in week 12 of semester, except for question 3 (function of the condenser) which was answered correctly by a significantly greater proportion of the class in 2009 (chi-square, χ2 8.3, d.f. 1, p<0.01). At the end of semester 1 in 2008 and 2009, a majority of students knew in theory where the magnification lenses were (question 1) and that the 40x objective was never used for initial observations (question 6). In comparison, students could not recall the correct order of steps to set up a microscope (question 5). It was also clear that students were confused about the function of the condenser and iris diaphragm (questions 2, 3, 4). In-class observations confirmed that students rarely adjusted the condenser and iris diaphragm when using the microscope. Table 1: Percentage of students who responded correctly to the quiz question. Symbols denote a significantly greater number of correct responses (within years) at the significance level of p<0.001(*) and p<0.05 (#). Questions Responses Responses 2008 2009 Section 1 – Essential parts of the microscope March

n = 282 May n = 200

March n = 305

May n = 318

1. Which microscope components contain lenses that are involved in producing the magnified image of the specimen?

53 66* 25 70*

2. Which microscope component regulates the depth of focus of the microscope?

10 11 6 13*

3. Which microscope component is used to focus light onto the specimen?

20 27# 17 34*

4. Which components of the microscope must, because of their function, be located between the light source and the specimen?

37

50*

14

50*

Section 2 – Using the microscope 5. What is the most appropriate sequence of the listed steps for setting up the microscope for specimen examination? Step 1. Securing and centering the specimen Step 2. Ensuring adequate clearance between the stage and the objectives Step 3. Positioning and focusing the specimen Step 4. Ensuring proper illumination of the specimen

19

23

21

25

6. Which objective is never used for initial examination of a specimen? 79 78 47 75*

In-class observations were made of 34 students in 2008 and 13 students in 2009. In both years, all or nearly all students placed the slide on the stage correctly, and began their examination with a low power lens (Table 2). In both years, very few students attempted to keep both eyes open whilst viewing the image with the microscope which may result in eye strain if microscopes are used regularly (Table 2). A clear majority of students did not correctly set the condenser or use the iris diaphragm whilst examining a specimen in either year (Table 2). Only 25 students (<1% of the 2009 class) responded to the request for feedback about the effectiveness and design of the online module. Even so, some interesting and helpful feedback was gained. Aspects of the online module which students identified as important contributors to their learning of microscopy were the content "… clarification of which parts magnify and how to focus", the design "The easy, understandable layout that allows you to choose which segment of the module you would like to go over", the interactive nature of the module "The ability to zoom in or zoom out on specific components of the microscope and information" and the problem solving exercises. Suggestions for improving the online module included the opportunity to pause during explanations and better integration or linking with the subsequent practical classes which require students to use microscopes to examine specimens.


16

Table 2: Percentage of students who were observed completing each task in 2008 and 2009.

Tasks 2008 2009 Student checks the slide before securing it on the stage 100 100 Student secures the slide on the stage adequately 100 100 Student begins specimen examination with the x4 or x10 objective 97 93 Student attempts to keep both eyes open when examining a specimen 12 8 Student correctly sets the condenser and the iris diaphragm 15 0

DISCUSSION Overall, the online Introduction to Microscopy module achieved better learning outcomes than the laboratory exercise. This finding agrees with other studies that have shown that virtual laboratory exercises, such as dissections and experiments, can result in conceptual learning outcomes equivalent to wet laboratory exercises (Kinzie et al., 1993; Hughes, 2000; Franklin et al., 2002). Despite this result, the quiz revealed gaps in knowledge in both years, particularly regarding understanding of the function of the condenser and iris diaphragm, and the correct order of steps to set up a microscope. In-class observations confirmed that students rarely adjusted the condenser and iris diaphragm when using the microscope. Allocating time to practice laboratory skills such as microscopy is essential for developing competency, and students are often not given the opportunity to develop that skill to a level where it can be used effectively (Peat & Taylor, 2004). It has been argued that better learning outcomes could be achieved if skills are taught independently before students are required to apply those skills to new problems (Friedler & Tamir, 1986; Johnstone & Letton, 1988-89; Johnstone & Wham, 1982). The ADBE online module introduces the microscope components and their functions, and provides some of the background information needed to understand how a microscope works. The use of microscopes in subsequent laboratory classes reinforces what students learn in the introductory training and allows students to practice the skill, but in the current practical timetable for ADBE, the microscope is not used for several weeks after completion of the introductory module. The link between the introductory online module and microscope use in class could be consolidated with a short focused laboratory exercise (which gets students to use the knowledge they've just gained) scheduled immediately after completion of the online module. The online problem solving exercises were identified as important contributors to student learning of microscopy by some students, so the complementary laboratory exercise could include a set of practical problem solving exercises e.g. a comparison of two different microscope slides requiring different condenser and iris diaphragm settings. Several students were positive about the design and interactive nature of the module, indicating that students were engaging positively with the ICT learning resource as found by James et al., (2010). The effectiveness of the online module depends on its integration into the rest of the practical program and provision of many opportunities for students to practice the skill as well as consolidate the theoretical learning. Embedding the IM module in the curriculum of all four first year biology subjects will streamline the teaching of microscopy at La Trobe University and provide additional opportunities for students to practice and consolidate their knowledge of how to use a microscope. Flexible delivery of this learning resource (via the University's learning management system) will allow students to revise as needed through the year.

ACKNOWLEDGMENTS Many thanks to the La Trobe University Flexible Teaching and Learning Team for their assistance with developing and revising the online Introduction to Microscopy module, particularly Darren Britten, Craig Coster and Karli Karvelas. Thanks to Tania Blanksby for assisting in the early stages of development of the module and Thea Shell for providing constructive and helpful feedback on version 1.0. This project was funded by a School of Life Sciences Teaching and Learning Development Grant and was approved by the La Trobe University Human Ethics Committee (Project number: FHEC08/R3).

REFERENCES Downie, R. & Meadows, J. (1995). Experience with a dissection opt-out scheme in university level biology. Journal of Biological Education, 29, 187-194. Franklin, S., and Peat, M. (1998). Online learning: the first year biology way. Proceedings of ASCILITE Conference 1998, 241-

249.


17

Franklin, S., Peat, M., & Lewis, A. (2002). Virtual versus traditional dissections in enhancing learning: a student perspective. Journal of Biological Education, 36, 124-129.

Friedler, Y. & Tamir, P. (1986). Teaching basic concepts of scientific research to high school students. Journal of Biological Education, 20, 263-269.

Gorst, J. & Lee, S. (2005). The undergraduate life sciences laboratory: Student expectations, approaches to learning and implication for teaching. In C.McLoughlin & A. Taji (Eds.) Teaching in the sciences. Learner-centred approaches. New York: Food Products Press.

Harris, T., Leaven, T., Heidger, P., Kreiter, C., Duncan, J. & Dick, F. (2001). Comparison of a virtual microscope laboratory to a regular microscope laboratory for teaching histology. The Anatomical Record (New Anat.), 265,10-14.

Hughes, I. E. (2000). Alternatives to laboratory practicals - Do they meet the needs? Innovations in Education & Teaching International, 38, 3-7.

James, R., Krause , K., & Jennings, C. (2010). The first year experience in Australian universities: findings from 1994 to 2009. The University of Melbourne: Centre for the Study of Higher Education.

Johnstone, A. H. & Letton, K. M. (1988-89). Teaching the Large Course: Is practical work practicable? Journal of College Science Teaching, 18, 190-192

Johnstone, A. H. & Wham, A. J. B. (1982). The demands of practical work. Education in chemistry,19, 71-73. Kinzie, M.B., Strauss, R., & Foss, J. (1993). The effects of an Interactive Dissection Simulation on the Performance and

Achievement of High School Biology Students. Journal of Research in Science Teaching, 30(8), 989-1000. Krause, K. & McEwen, C. (2009). Engaging and retaining students online: a case study, in The Student Experience,

Proceedings of the 32nd HERDSA Annual Conference, Darwin, 6-9 July 2009: pp 251-262. Meester, M. A. M. & Maskill, R. (1995). First-year chemistry practicals at universities in England and Wales - aims and the

scientific level of the experiments. International Journal of Science Education, 17, 575-588. Peat, M. (2000). Towards First Year Biology online: a virtual learning environment. Educational Technology & Society, 3(3),

203-207. Peat, M., & Franklin, S. (2001). Managing change: the use of mixed delivery modes to increase learning opportunities.

Australian Journal of Educational Technology. 17(1), 37-49. Peat, M. & Taylor, C. (2004). Virtual biology: How well can it replace authentic activities? Synergy, 20, 25-27. Saunders, W. L. & Dickinson, D. H. (1979). A comparison of community college students' achievement and attitude changes in

a lecture-only and lecture-laboratory approach to general education biological science courses. Journal of Research in Science Teaching, 16, 459-464.

Stuckey-Mickell, T.A., & Stuckey-Danner, B.D. (2007). Virtual Labs in the Online Biology Course: Student Perceptions of Effectiveness and Usability MERLOT Journal of Online Learning and Teaching. 3(2), 105-111.


18

COLLABORATIVE LABORATORY FOR QUANTITATIVE DATA ANALYSIS Adam J. Bridgeman, Siegbert Schmid Presenting author: Adam J. Bridgeman ([email protected]) School of Chemistry, University of Sydney, Sydney NSW 2006, Australia

KEYWORDS: collaborative learning, quantitative analysis, generic attributes, large classes

ABSTRACT In this project, students share experimental results to perform data analysis and to develop an appreciation of precision, accuracy and reliability of experimental data and of the scientific method. The number of students taking Junior Chemistry means that the data sets are large and naturally contain random, systematic, and even deliberate errors. By forcing students to work with a wide range of measurements including their own, students develop an appreciation of the importance of the role of human error in the physical sciences. In doing so and in using spreadsheet software, key generic attributes including quantitative, problem solving and inquiry skills are developed and deficiencies in the computer skills are addressed. The project has led to real improvements in the development of generic attributes in our courses, at minimal expense. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 18-23, ISBN number 978-0-9808597-0-6

INTRODUCTION Graduate attributes are those skills and qualities that students should possess alongside discipline knowledge upon graduation (Bowden, Hart, King, Trigwell & Watts, 2000). There are generic graduate attributes that should be achieved by all graduates, irrespective of their degree program. Universities use these qualities to differentiate their graduates and to increase their marketability to prospective employers because they are deemed “work ready” and capable of ongoing learning and development (Barrie, 2007). By stressing the employability of its graduates, a university can in turn seek to make a distinction when marketing their courses. Science graduates, however, should acquire additional specific generic and science procedural skills (Jones, 2009). They should be trained in and ready to apply the scientific method in their professional lives. It is just these quantitative, problem solving and inquiry skills that make science students so employable in a variety of graduate careers outside science (Jones, Dermoudy, Hannan, James, Osborn & Yates, 2007). Science graduates should possess the employability skills required in a competitive and global employment market (DEST, 2002). Whilst subject specific knowledge is important in research and a narrow range of other careers, the ability to tackle new problems using the scientific method is the key attribute that science graduates can also use to differentiate themselves in other careers. The types of skills and attributes that are distinctive of the scientific method are (or should be) an integral and explicit enabling component of the science degree (Barrie, 2007). However, these transferrable skills should be central learning outcomes not only for students who graduate in scientific disciplines but also for the many more who only take Junior Science units. For such students, the exposure to the scientific method and to the ways in which scientists work is arguably much more important than the acquisition of subject specific knowledge. In the design of a Junior Science unit, the overriding facet is commonly its preparation for the Intermediate Science units for which it is a pre-requisite. This often translates to a focus on subject specific, content ‘dot points’ and a first year of the BSc degree which is extremely content heavy (Leggett, Kinnear, Boyce & Bennett, 2004). There is usually considerably less focus on the development of quantitative, problem solving and inquiry skills leading to an uneven skill set in Second Year students (Green, Hammer & Star, 2009). The development of these skills occurs in the 3rd and 4th years of the degree where there is a focus on research projects, and many of the attributes may only be taught at a foundation level in the first year curriculum (Barrie, 2006; Peat, Taylor & Franklin, 2005).

Bridgeman, Schmid, Collaborative Laboratory for Quantitative Data Analysis

19

The rationale for the implicit introduction and development of the scientific method in Junior Science thus covers both students who will ultimately graduate as scientists and those taking science units as part of a professional degree, for interest or simply to collect credit points. Many first year students do not recognise the generic skills that are developed in first year science units and do not appreciate where their deficiencies in these areas may lie. This can lead to low student engagement in activities specifically designed to develop the scientific method, including laboratory work (Leggett, Kinnear, Boyce & Bennett, 2004). As a result, course and unit evaluation scores pertaining to graduate attributes are, perhaps not surprisingly, traditionally low. This paper describes one initiative in our ongoing attempts to embed graduate attributes and to develop the scientific method in our Junior Chemistry units. Its aim is to develop skills in data analysis, team work and information technology and to develop an appreciation of how real science works. It seeks to take advantage of the large number of students taking first year Chemistry to provide a large database. It also utilises the range of abilities and attitudes of these students to highlight that even physical sciences like chemistry are human activities (Mahaffy, 2006) and hence open to all the failings and biases of the humans that are engaged in it.

METHODS AND RESULTS A large number of students, around 2000 students in semester 1 and around 1600 students in semester 2, take Junior Chemistry units at The University of Sydney. We provide 8 units in semester 1 and 7 units in semester 2, including service units for Pharmacy, Veterinary Science and Medical Science degrees and 4 levels of units taken by students in other faculties, including Science. These 4 levels (“Fundamentals”, “Mainstream”, “Advanced” and “Special Studies”) allow us to design units for specific levels of prior knowledge. In many of our laboratory classes, students record measurements. In this project, all such data is pooled. The students will then download the measurements from around 2000 separate observations and manipulate them. In doing this, they develop skills in dealing with large datasets including graphing and presenting data, statistical analysis and dealing with outliers. They collaborate with a large number of fellow scientists and, hopefully, come to appreciate the meaning of accuracy, precision and significance in numerical results by making value judgements about the reliability and relevance of information in a scientific context. In this project, data collection and analysis has been performed on 4 experiments. In each case, the experimental instructions were unchanged compared to those used before the intervention. After recording their measurements at the bench, each student enters their data via an online form. The data was collected in this way over the course of the week and automatically released on Friday afternoon in the cross-platform csv (comma separated values) format for students to download and open in any spreadsheet software. The data analysis is performed entirely using the spreadsheet software. The instructions are deliberately tailored towards Excel because this is the spreadsheet software available in the university computer rooms and library and because this is the most common data analysis software used in workplaces. However, all of the tasks can also be tackled using other spreadsheet programs, including Apple iWork and the free OpenOffice and Google documents. Below, we outline the procedures and results for 2 of the 4 experiments. In the ‘Properties of Gases’ experiment, students obtain the molar mass of an unknown gas in two ways: from density and from flow times measurements. Figure 1 shows the distribution of masses obtained in semester 1 of 2010. As can be seen from the figure, the measurements obtained by the students naturally fit a Gaussian distribution centred on the molar mass of carbon dioxide. The main aim of asking students to plot this data was to illustrate how a normal distribution arises even in a relatively simple experiment performed under apparently identical conditions. Students were asked to consider and comment on the precision and accuracy of the two experimental methods using both the graphical results and through calculation of the averages, standard deviations and standard errors. Students were asked to report the unknown mass including a justification of the number of significant figures based on the analysis.


20

Figure 1: Distribution of experimental molar mass masses from (a) flow times experiment (●) and (b) density experiment (×) obtained by Junior Chemistry students in semester 1 of 2010. During the assessment period for this task, it was clear from discussion board activity and from an informal focus group that a significant proportion of students do not know how to use the basic functions of spreadsheet software, such as drawing simple charts. It appears that despite such software being available in high schools and despite being both relatively high achievers and reasonably motivated, these “generation-Y” students manage to avoid such exercises at school. Clearly, “the assumption that some Academic staff make, that students already have computer literacy skills, is often erroneous.” (Reid, 1997). This assumption is certainly one that we made and is common in many science units. As a result, students are now given access to targeted support in Excel using the exSite available on the learning management system. It is also clear that, whilst many students lack necessary computer skills, few will attend separate courses. The teaching of computer skills is more effective when built into academic subjects and assessment, rather than as an add-on (Link & Marz, 2006). It is also important to design activities that help to solve practical problems and show the benefits of using computers in order to engage students from the whole spectrum of computer literacy. As noted above, a consequence of using a data set in the analysis which is real and large is that it naturally fits a normal distribution. Of course, another consequence is that it contains points which are suspect. No doubt, this may be due to data entry errors, a failure of equipment or incompetence. Another source of outliers is “fraud”. The data entry is deliberately designed to be anonymous, ideally to ensure that students enter their actual results. However, it also, in a large class, invites a number of students to intentionally submit wrong (and often ridiculous) results. These results are not removed from the data set prior to its release to students. Instead, students are confronted with such factors as a reality of the scientific endeavour. The methods used to assess the reliability of individual measurements are introduced and students use Chauvenet's criterion (Taylor, 1997) as a statistical indicator of reliability. In the ‘Enantiomers - Chirality in Organic Chemistry’ experiment, students measure the optical rotation of the isomers of limonene at different concentrations. The equipment used for this is really designed to illustrate the general principle that optically active molecules rotate polarised light and that a pair of enantiomers do so in opposite directions. In the experiment, each student therefore records the rotation of light by each enantiomer at a single concentration. Repeating the experiment multiple times at different concentrations is required to obtain a meaningful numerical result but is not a task that a typical student enjoys and is irrelevant to the main learning outcome of the actual experiment. Such a task is, however, ideally suited as a large group activity and, over a week, the data set contains multiple measurements at each concentration.

0

50

100

150

200

250

300

‐10 40 90 140 190

molar mass (g mol‐1)


21

Using the linear regression facility of Solver within Excel, the students obtain the specific rotation of limonene. Figure 2 shows the result of this analysis obtained in semester 2 of 2009. The gradient and its standard error are used by students to calculate the specific rotation of limonene. In the 4 occasions that this analysis has been performed using large classes, the value obtained is close to the literature value despite the scatter in the data and the simplicity of the experimental set up. On 2 occasions, the analysis was performed using a small class of less than 50 students and this leads to poor results. The analysis uses a relatively advanced feature of Excel to show how information can be extracted from experimental data, and the value of repeating experiments.

Figure 2: The dependence of the optical rotation on concentration of (+)-limonene (×) and (-)-limonene (●) obtained by Junior Chemistry students in semester 2 of 2009. The task is completed by students analysing the residuals, the difference between the measured optical rotation and that calculated using the value for the specific rotation, obtained from the regression. Figure 3 shows the residuals for the (-)-enantiomer from the same analysis as shown in Figure 2. The distribution of the residuals is an indication of the nature of the error in the experiment. Typically, the residuals for this experiment suggest that there is a bias towards overestimating rotation at low concentrations and underestimating it at higher concentrations. The students are asked to assess this and hence comment on ways in which the experimental set up can be improved.

Figure 3: Residuals obtained from the linear regression for (-)-limonene obtained by Junior Chemistry students in semester 2 of 2009.

‐60

‐40

‐20

0

20

40

60

0 5 10 15 20 25 30 35

concentration (%)

rotation(°)

‐40

‐30

‐20

‐10

0

10

20

30

40

0 5 10 15 20 25 30 35Residuals

concentration (%)


22

Figure 4 shows the student evaluation scores for the question “This unit of study helped me develop valuable graduate attributes” on the unit of study evaluation for a selection of our units. The figures for 2008, before the introduction of the work described in this paper, are consistent with those from previous years, and with qualitative comments by students. Although the figures for 2009 and 2010 suggest that we still have much to do, they show a consistent and pleasing improvement across all units. It should be noted that the students were made directly aware of the importance of graduate attributes in the course and the particular skills which this activity seeks to develop.

Figure 4: Student evaluation scores relating to graduate attributes (out of 5) for science (CHEM1001 – Fundamentals, CHEM1101 – Mainstream and CHEM1901 – Advanced) and service units (CHEM1108 – Life Sciences and CHEM1611 – Pharmacy). All students were invited to complete evaluations with around 300 choosing to do so in each of the years shown.

CONCLUSIONS The primary aim of the project was to improve the ability of students to perform data analysis and their appreciation of the precision, accuracy and reliability of experimental data. This formed part of a general strategy to develop generic attributes more thoroughly in our courses and to address deficiencies in the computer skills of our students. Using a relatively simple web interface, students’ experimental results are collected leading to large data sets for the subsequent analysis which naturally include random, systematic, and even deliberate errors. These factors are often not appreciated in undergraduate classes in the physical sciences where there is traditionally a strong desire to reproduce a theoretical result. This motivation is completely against the way science is conducted at research level and opposes the critical skills that our graduates should possess. This project aims to confront this attitude by developing an inquiry and student-centred approach to the analysis of data. The project is also built around the role of peer interaction in science. By forcing students to work with a wide range of measurements including their own, students will appreciate the importance of the role of human error in the physical sciences. The project has successfully met its objectives of improving the quantitative, problem solving and inquiry skills and computer literacy in Junior Chemistry students. Student feedback has been positive, with a noticeable increase in the percentage of students finding the task useful for developing valuable graduate attributes.

ACKNOWLEDGEMENTS This project was funded through The University of Sydney ‘Teaching Improvement and Equipment Scheme’.

REFERENCES Barrie, S.C. (2006) Understanding what we mean by generic attributes of graduates, Higher Education 51(2) 215-241 Barrie, S. C. (2007) A conceptual framework for the teaching and learning of generic graduate attributes. Studies in Higher

Education, 32(4), 439-458. Bowden, J., Hart, G., King, B., Trigwell, K. & Watts, O. (2000) Generic capabilities of ATN university graduates. Retrieved 17

June 2010 from http:/www.clt.uts.edu.au/ATN.grad.cap.project.index.html DEST (2002) Employability Skills for the Future, DEST, Canberra. Retrieved 17 June 2010 from

0

0.5

1

1.5

2

2.5

3

3.5

4

All CHEM1001 CHEM1101 CHEM1901 CHEM1108 CHEM1611

2008

2009

2010


23

http://www.dest.gov.au/sectors/training_skills/publications_resources/profiles/documents/final_report_x1_pdf.htm Green, W., Hammer, S., and Star, C. (2009) Facing up to the challenge: why is it so hard to develop graduate attributes?

Higher Education Research and Development, 28 (1). 17-29. Jones, A. (2009) Generic Attributes as espoused theory: the importance of context, Higher Education 58(2) 175-191 Jones, S.M. Dermoudy, J., Hannan, G., James, S., Osborn, J. and Yates, B. (2007) Designing and mapping a generic

attributes curriculum for science undergraduate students: a faculty-wide collaborative project, Proceedings of the UniServe Science conference October 2007 Sydney.

Leggett, M., Kinnear, A., Boyce, M. and Bennett, I. (2004) Student and staff perceptions of the importance of generic skills in science. Higher Education Research and Development, 23, 295–312.

Link, T.M. and Marz, R. Computer literacy and attitudes towards e-learning among first year medical students, BMC Medical Education, 6(1), 34-43.

Mahaffy, P (2006) Moving Chemistry Education into 3D: A Tetrahedral Metaphor for Understanding Chemistry, Journal of Chemical Education, 83(1), 49-55.

Peat, M., Taylor, C.E. and Franklin, S. (2005) Re-engineering the undergraduate science curriculum to emphasis the development of lifelong learning skills, Innovations in Education and Teaching International, 42(2), 10–21.

Reid, I, (1997) Computer Literacy in Higher Education, Proceedings, ASCILITE97 Conference. Taylor, J. R. (1997) An Introduction to Error Analysis. 2nd edition, Sausolito, California: University Science Books, 166-8.


24

TRAINING ETHICAL SCIENTISTS: STUDENT VIEWS ON THE BENEFITS OF USING ANIMALS IN LEARNING Ashley Edwards, Susan M. Jones Presenting author: Ashley Edwards ([email protected]) School of Zoology, University of Tasmania, Hobart Tas 7001, Australia

KEYWORDS: animal-based activities, ethical awareness, student perceptions, zoology teaching

ABSTRACT The use of animals in science teaching can create an environment which enhances learning outcomes, providing opportunities for students to add value by engaging with authentic experiences, but also offering the chance to explore the ethical issues surrounding the use of animals in teaching, learning and research. This paper describes the benefits perceived by undergraduate Zoology students of exposure to animals in their learning. Results demonstrated that students value highly the chance to work with animals during their undergraduate careers, and that the nature of this appreciation changes and matures as they develop into more independent learners. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 24-28, ISBN Number 978-0-9808597-0-6

THE ISSUE The use of animals in biology teaching in universities worldwide is becoming more controversial (Franklin, Peat & Lewis, 2002). Indeed, there are some tertiary institutions in the UK which have removed the use of animals, or offer the chance to “opt out” of animal-based activities, in degree programs such as veterinary science (Heron 1992). These views are a reflection of the current perceptions of society (Hagelin, Carlsson & Hau, 2000; Wheeler, 1993), including students enrolled in courses which incorporate the use of animals in teaching and learning (Mangan, 2000; Pederson, 2002).

THE BACKGROUND Advocates of the benefits of the use of animals in teaching suggest that while alternative approaches can be appropriate in some circumstances, there are a number of situations in which there is simply no substitute for the use of animals or animal tissues (Pederson, 2002; Wheeler, 1993). The benefits of using animals, tissues, or preserved specimens include the following: 1) Having an authentic experience – there is substantial published literature attesting to the benefits that the authenticity (relevance and realism) of an experience can have on learning outcomes (see Herrington & Herrington, 2006 and references therein); 2) Understanding dynamic and interactive systems – static examples such as diagrams, models and computer simulations simply cannot render these experiences accurately (Pederson, 2002); 3) Tapping into multiple learning modes – it is certainly well established that many students engage and learn more effectively when given the opportunity to investigate, to actively participate in learning activities and to gain hands-on experience which complements theoretical learning (Mayer, Bove, Bryman, Mars & Tapangco, 1996); 4) Building an appreciation of ethical and animal welfare issues – working with animals affords the important opportunity to discuss animal ethics issues with students, and allows the students to develop and defend a position (Pederson, 2002).

THE UTAS SCAFFOLDED APPROACH In the School of Zoology at the University of Tasmania we have designed a vertically integrated approach to allow our students to develop an appreciation of animal ethics across the three years of the undergraduate course. This program aims to maximise learning outcomes whilst simultaneously

Edwards, Jones, Training Ethical Scientists

25

addressing several of the UTAS Generic Graduate Attributes (GGAs); knowledge, problem-solving skills and social responsibility (UTAS GGA policy, 2001). Our program includes the supervised handling and observation of live vertebrate and invertebrate animals, dissection of dead animals and inspection of preserved specimens. Students are asked to kill an invertebrate (sea urchin) by immersion in warm fresh water in their 2nd year, and mice are killed while students are in the room in their 3rd year. In all cases students are offered the opportunity to undertake the level of involvement with which they are comfortable; this ranges from full participation, to watching another student, to completing an alternative task at another time. We value, and want our students to value, the use of animals in their learning experiences and, therefore, relevant assessment tasks are embedded in our learning curriculum. The University of Tasmania Animal Ethics Committee requires that activities involving animals are appropriately assessed and explicitly address stated learning outcomes. Furthermore, the Australian Code of Practice for the Care and Use of Animals for Scientific Purposes (hereafter: The Code) establishes that all teaching activities for which animals are used are required to clearly demonstrate that learning outcomes cannot be achieved in another way, and that every attempt has been made to observe the “3Rs” – Replace animals with an alternative, Reduce the number of animals used and Refine techniques and procedures, whenever possible (NHMRC, 2004). The Code provides that all students are “given the opportunity to discuss the ethical, social and scientific issues involved in the use of animals for scientific purposes, including teaching” (NHMRC, 2004). In the School of Zoology, University of Tasmania, observance of the requirements set out in The Code begins in 1st year, when students are introduced to the ethical framework that guides the use of animals in teaching and research. Students are also given the opportunity to discuss the advantages and disadvantages of, as well as their own views about, the use of animals in their learning experiences. They also consider relevant aspects of scientific method and experimental design such as appropriate types of animals and sample sizes for experiments. In 1st year practical classes, students examine or dissect a number of invertebrate specimens, and to improve their ability to take full advantage of these learning opportunities, we offer a “pre-Lab” on line learning opportunity to enhance prior knowledge and minimise the possibility of cognitive overload during the practical session (Cook 2006; Jones & Edwards, 2010 (forthcoming)). In 2nd year, students are given their first opportunity to work with vertebrates and cephalopod molluscs in the field and the laboratory. Their roles and responsibilities under The Code (NHMRC, 2004) are discussed in class, and each student signs the Student Declaration. The value of student learning on field trips has been clearly demonstrated in the areas of engagement, motivation, and informing future decisions (Prokop, Tuncer & Kvasničák, 2007). In 3rd year, students must take a greater personal responsibility for the care and use of animals. We have, therefore, designed specific learning tasks through which students develop a professional level of awareness of the processes of gaining animal ethics approval for scientific research. Students undertake animal husbandry responsibilities, and, under close supervision, design and undertake research projects applying rigorous experimental design and ethical guidelines, both in the field and the laboratory.

RESULTS We evaluated student perceptions of the use of animals in our teaching program to examine whether our approach is helping students to gain an appreciation of the advantages of such opportunities. During 2009, we surveyed students at 1st (end of Sem 1 and end of Sem 2), 2nd (end of yr) and 3rd year (end of Sem 1 and end of Sem 2) and asked them a series of questions designed to explore their perceptions of the benefits to their learning of using animals.


26

Figure 1: Proportion of total comments of students who perceived the use of animals or tissues to be extremely helpful to their learning, by year level (black). Number of students responding to the survey indicated in parentheses. First and third year comments are also separated into Sem 1 (grey) and Sem 2 (white) responses. To investigate students’ appreciation of the use of animals in their learning, we asked “Do you feel that your learning experience was improved by the use of animals or animal tissues” (four Likert scale options ranging from Extremely to Not at all). We saw a steady increase in the proportion of students at each year level who responded with “Extremely” helpful to their learning (Figure 1, black bars). There was a statistically significant increase between years ((d.f = 2) = 58.800, p < 0.0005). We interpreted this result as students showing an increasing appreciation for these opportunities as they matured as learners. Further, when we examined the effects of our 1st yr interventions more closely, we observed that the proportion of students who responded in this way had indeed increased significantly ( (d.f. = 1) = 32.175, p < 0.0005) from 1st to 2nd semester (Figure 1, grey and white bars). Excitingly, 3rd year students also demonstrated a significant increase in the number of students reporting the use of animals as extremely helpful to their learning ( (d.f. = 1) = 12.991, p < 0.0005). To examine the nature of this appreciation for learning opportunities involving animals we asked “What benefits do you see or expect to gain from the use of animals or animal tissue in your learning in Zoology”? Responses were assigned to one of five categories, and we examined whether the relative emphasis on a particular type of benefit varied during their progress through our undergraduate program (Table 1). Results are presented as proportions of the total number of comments made, and demonstrate that during their 1st and 2nd years of study, students were most focused on the authenticity of the learning experiences (62.8 and 57.9 %, respectively), wanting them to be realistic and engaging in order to more strongly reinforce theoretical concepts. This group of responses is typified by the following student comments: “It’s hard to form a real interest for the subject area without any interaction with real animals” “Being able to see and examine (first hand) these animal/animal tissues cannot compare to photos or diagrams. Real-life usage of these tissues allows the experience to be unforgettable and thus, allows me to remember animal structures better” However, by 3rd year, fewer (35.6 %) student comments identified “bringing authenticity to a learning experience” (Table 1) as a benefit of working with animals. Instead, the proportion of comments indicating the importance of seeing the relevance of current learning to future study or career opportunities increased significantly ( (d.f. = 1) = 17.710, p < 0.0005) from 11.5 % in 1st year to 26.1 %. Further, comments relating to the students’ learning mode and the opportunity to gain hand-on experience rose from 20.0 % in 1st year to 27.6 % by 3rd year. As exemplified below, students felt they gained: “Knowledge about the animals i.e. what happens to them, the respect they receive and the skills learned can all contribute to employment - a greater understanding will help make more appropriate decisions”

0

10

20

30

40

50

60

70

80

1st yr (N=266) 2nd yr (N=56) 3rd yr (N=51)

"ext

rem

ely"

hel

pful

to

lear

ning

(%

)

Student year level

Edwards, Jones, Training Ethical Scientists

27

Table 1: Proportion of total comments in response to the open-ended question “What benefits do you see or expect to gain from the use of animals or animal tissue in your learning in Zoology?”

Benefit Type of benefit End of 1st yr (%)

End of 2nd yr (%)

End of 3rd yr (%)

1 Bringing authenticity to a learning experience 62.8 57.9 35.6 2 Seeing relevance of current learning to future 11.1 17.1 26.1 3 Wanting to help others/animals/environment* 4.5 5.3 10.8 4 Relating to learning mode - seeing, hearing, doing, 1st hand

experiences 20.0 19.7 27.6

5 Few or none 1.7 0 0 *Includes the opportunity to develop awareness of animal ethics issues, aligned with UTAS GGA of Social Responsibility To investigate the development of students’ ethical awareness, we asked “Were you given any verbal/written/online information about animal ethics at the time the animals or animal tissues were used?” Figure 2 shows that the proportion of students responding positively to this question also increased significantly between years (( (d.f. = 2) = 35.355, p < 0.0005), particularly from semester 1 (47.0 %) to semester 2 (66.0 %) in 1st year (grey and white bars), and then again by the end of the 2nd year (80.0 %) of our teaching program, showing that our scaffolded strategy does increase student awareness that animal ethics issues exist. Similarly, the proportion of comments relating to the University of Tasmania’s GGA of Social Responsibility* (Table 1) doubled from 1st to 3rd year in students surveyed (4.5 to 10.8 %) and the survey also elicited some reflective comments which endorsed our teaching approach: “Using animals makes us think more ourselves about the animal ethics issues, which are likely to be important in our futures in this field” “At every stage animal ethics [issues] are drilled in. No one in the School of Zoology at UTAS could possibly take this for granted”

Figure 2: Proportion of students who recalled being given information relevant to animal ethics at the time of the activity, by year level (black bars). Number of students responding to the survey indicated in parentheses. First and third year comments are also separated into Sem 1 (grey) and Sem 2 (white) responses.

CONCLUSION Our results show that as undergraduate students matured as learners, their perceptions of the advantages to their learning gained from the use of animals shifted, from the benefits of hands-on experiences for successfully completing the task at hand (1st year), towards the cumulative contribution those experiences were making toward their future learning and careers (3rd year). Students themselves appreciate and gain high satisfaction from hands-on authentic activities in biology laboratories (Peat & Taylor, 2005; our own survey data (Table 1)). Final year students in the present study also demonstrated an increased appreciation of the importance of developing an awareness of relevant animal ethics issues. Our findings are supported by previous research which demonstrated proportion of students in favour of animal-based experiences increased with year level (Smith, 1994). Students are perhaps better prepared to benefit

0

20

40

60

80

100

1st yr (N=266) 2nd yr (N=56) 3rd yr (N=51)

"yes

" re

spon

se

(%)

Student year level


28

from these experiences later in their time at university (Downie & Alexander, 1989; Hagelin et al., 2000). With University animal ethics committees increasingly demanding justification that learning objectives cannot be met by the use of alternatives such as computer simulations, videos or models (Predavec, 2002), educational goals need to be carefully considered when deciding if the use of animals is necessary (Hagelin et al., 2000). Our study draws attention to the need for careful planning in accordance with the rigorous framework of The Code. As teachers, we need to create and implement animal-based activities which simultaneously enhance student learning outcomes and afford students the opportunity to comply with their responsibility under The Code to gain maximum benefit from the learning opportunities offered to them.

ACKNOWLEDGEMENTS This work was supported by a UTAS Teaching Development Grant and conducted under Tasmanian Social Sciences Human Research Ethics Committee approval number H0010485. We thank Dr Ruth Casper for help with data entry.

REFERENCES Cook M. P. (2006) Visual representations in science education: the influence of prior knowledge and cognitive load on

instructional design principles. Science Education, 90(6), 1073-1091. http://www3.interscience.wiley.com/cgi-bin/fulltext/112657494/PDFSTART

Downie R. and Alexander L. (1989) The use of animals in biology teaching in higher education. Journal of Biological Education, 23(2), 103-111.

Franklin S., Peat M., and Lewis A. (2002) Traditional versus computer-based dissections in enhancing learning in a tertiary setting: a student perspective. Journal of Biological Education, 36(3), 124-129.

Hagelin J., Carlsson H. E. and Hau J. (2000) The importance of student training in experimental procedures on animals in biomedical education. Scandinavian Journal of Laboratory Animal Science, 27, 35-41.

Herrington A. and Herrington J. (2006) Authentic Learning Environments in Higher Education. Information Science Publishing, Melbourne, Australia.

Heron L. (1992) Cutting out the cutting up: Vivisection and animal dissection are becoming optional in some universities. The Independent, 17 Dec 1992. http://www.independent.co.uk/news/education/education-news/cutting-out-the-cutting-up-vivisection-and-animal-dissection-are-becoming-optional-in-some-universities-says-liz-heron-1564015.html

Jones S. M. and Edwards A. (2010) Online pre-laboratory exercises enhance student preparedness for first year biology practical classes. International Journal of Innovation in Science and Mathematics Education. 18(2) (forthcoming)

Mangan K. S. (2000) Can vet schools teach without killing animals? The Chronicle of Higher Education, 46, A53-A54. http://jobs.chronicle.com/article/Can-Vet-Schools-Teach-Witho/10357/

Mayer, R. E., Bove, W., Bryman, A., Mars, R. and Tapangco, L. (1996) When less is more: meaningful learning from visual and verbal summaries of science textbook lesons. Journal of Educational Psychology, 88, 64-73.

National Health and Medical Research Council (2004) Australian Code of Practice for the care and use of animals for scientific purposes. 7th Edition, Australian Government. http://www.nhmrc.gov.au/_files_nhmrc/file/publications/synopses/ea16.pdf

Foster, Aligning Learning and Assessment Through Adaptive Strategies in Tutorials

29

ALIGNING LEARNING AND ASSESSMENT THROUGH ADAPTIVE STRATEGIES IN TUTORIALS IN PHYSICS AT THE UNIVERSITY OF AUCKLAND

Graham Foster ([email protected]) Physics Department, The University of Auckland, Auckland 1142, New Zealand

KEYWORDS: OASIS, strategies, alignment, tutorials, assessment

ABSTRACT Continuous assessment in physics is important for students. It provides the development of mental models by revising and revisiting concepts. In Physics 120 “Physics of Energy” and Physics 150 “Physics of Technology” at the University of Auckland, we identified that during 2007 and 2008 there was a significant and increasing non-participation rate in assignments and tests. In 2009 strategies were implemented to improve participation by adapting tutorials to be more interactive and aligned to the on-line assignment assessments. There were four online OASIS assignment assessments spread through each course. One week prior to submitting the assignment assessment, six practice questions were given. These questions were similar to those questions in the interactive tutorials. Four OASIS questions were given with changed variables as the assignment assessment. After the third and fourth OASIS assignments there was a strong indication that alignment between tutorial questions and assignments encouraged more participation and completion of OASIS assignments. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, pages 29-34, ISBN Number 978-0-9808597-0-6

INTRODUCTION “Assessment is primarily concerned with providing teachers and/or students with feedback information which they need to interpret when answering the three feedback questions: ‘where am I going?’, ‘how am I going?’ and ‘where to next?’” (Hattie, 2003). Physics tutorials need to provide students with these answers through a series of structured, timely questions that enable students to answer those three questions. Physics tutorials, therefore, should be high-quality, formative learning experiences that support students in their learning process. Popham (2008) offers the definition of formative learning as “a planned process in which assessment-elicited evidence of students’ status is used by teachers to adjust their ongoing instructional procedures or by students to adjust their current learning-tactics.” Wren(2008) has indicated that if they use formative assessment to benefit all their students “teachers must be willing to confront a number of obstacles including the willingness to reject the transmission model, the need to accept that students have an untapped potential for learning rather than a fixed learning potential.” Black and Wiliam (1998) mention other obstacles to learning related to assessment practices including two relevant to this paper: Grades are over-emphasised, while efforts to recognise student problems and provide useful

advice to students are not emphasizes enough Assessment feedback often results in students being compared with each other, which sends them

the message that they are in a competition O’Byrne and Thompson (2005) investigated the tutorial benefits of on-line assignments. They considered a list of conditions believed to promote student learning (Brown, Gibbs & Glover, 2003). Their investigation used an electronic resource MasteringPhysics, by Addison-Wesley. This resource was accessed by students through a web-page and was chosen because it offers advantages such as immediate feedback, immediate marking and personal login. These strategies reduced the chance of copying of assignments. Some gains were made over a paper-based assignment system. Staff ratings of the effectiveness of Mastering Physics assignments were complemented by comments, such as feedback was more ‘tailored to student responses’, ‘provided when required’, ‘sample solution does provide a good guide’ and ‘used to help answer problems’. Losses included the loss of group-work caused by students working under their own login name,


30

010203040

Physics 150 Percentage non‐participation

2007

2008

2009

since more students worked individually rather than in teams, and the alignment between questions in the paper-based assignments and those in the final examination. MOTIVATIONS At the University of Auckland there has been increasing non-participation in assignments and tests as the semester proceeded. Data collected from Physics 120 and Physics 150 from 2007, 2008 and 2009 shows that there has been very significant and increasing non-participation rates as the semesters developed, in both assignments and tests (see Figures 1 and 2) below.

Figure 1: Physics 120 Percentage non-participation rates 2007 to 2009 (2007 Test 2 data not available)

Figure 2: Physics 150 Percentage non-participation rates 2007 to 2009

This project was seeking to determine if increasing the quality of tutorials by increasing engagement and aligning tutorial questions to future assignment questions might improve the number of students completing assignments. THE OASIS ASSIGNMENT AND ASSESSMENT TOOL OASIS is a web-based learning and assessment tool. The Faculties of Science and Engineering at the University of Auckland use OASIS predominantly with first-year Physics and Engineering students for skills practice and summative assessment. The tool delivers individual tasks, marks student responses, supplies students with prompt feed-back by providing a response that indicates if the student’s response is correct or incorrect within a limit of accuracy (usually 1% to 2 %), provides the accepted answer together with a mark value (1/2, 1 or 2 marks per question) and logs student activity. The Physics 120 “Physics of Energy”, Physics 150 “Physics of Technology” and Physics 160 “Physics for Life Science” papers all have over 200 students per semester. Physics 160 attracts nearly 600 students in semester 2 each year as they compete towards entry into medical school. These large student numbers would result in a large marking load after each of the four assignments relevant to each paper. If paper assignments with hand-written answers were used then a large time commitment would be required for marking. OASIS relieves this marking load and provides better workload management for lecturers, together with instant feedback to students. In each of the above courses the four online OASIS assignment assessments are spread through each course. One week prior to submitting the assignment assessment, six on-line OASIS practice questions are given. The assessments include four of these questions given with different values of the same variables. Each student receives different values from other students. Smaill (2005) reports that first-year engineering students found the OASIS software easy to use as it provides all the data, requires no steps or reasoning to be given and clearly shows where the answers need to be entered. Smaill (2005) reports that OASIS helps them improve their skills and learning. This conclusion was based on extensive observation and analysis of assessment results. He concluded that the OASIS assessment system supported improved student achievement. The OASIS system presented no significant difficulties that would hinder achievement. DEVELOPMENTS At the University of Auckland students from a diverse range of backgrounds study within Physics 120 and 150, although very few are able to progress to Stage 2. In recent years to support first-year students, non-compulsory support tutorials have been provided for four hours daily. These are very


31

well attended in the few days before each OASIS assignment assessment occurs, but there is no focus to each tutorial. There are two tutors per hour and these change hourly. In addition, as part of the course work, there have been two-hour compulsory ‘laboratory tutorials’ alternating with three-hour laboratory experiments each fortnight. The assessments include assignments, tests, laboratory reports and examinations. In laboratory tutorials students are provided with questions from past examinations but there seems to be no relationship between the purpose of tutorials and the OASIS assignments.

THE LABORATORY TUTORIALS The tutorials are two hours each and it is intended that they should support learning and prepare students for assessments. In 2007 and 2008 students were provided with past paper questions from either tests or examinations in a study situation where tutor support is provided by post-graduate student tutors. These tutorials tended to become a very passive learning situation in which students would not always seek or receive the support they needed. In 2009 the structure for each tutorial in Physics 150 was changed to include two or three types of engagement. One initiative was to activate tutors to be pro-active helpers. Another initiative was to increase the level of student interaction so that more group work (using groups of 5 to 6 students) is implemented, thus increasing peer support and reducing the emphasis on competition for grades. A one hour training session for tutors was provided before the semester started. It was intended that this tutor development programme should enable tutors to guide the whole range of students towards problem solving more effectively than in previous years. A ‘Tutor Reply’ sheet was designed to provide feedback from tutors and students to the lecturers about difficulties students were having and the effectiveness of the tutorials. Tutorial 1: Questions similar to those used in the OASIS questions, taken from a variety of texts, were provided to the lecturers to confirm they were suitably focused. When answers were attached they were provided to laboratory tutors for the tutorials. Tutorial 2: Before the second tutorial the laboratory tutors were sent an email again reminding tutors of the strategies they might use to guide students towards problem solving using several steps. The tutorial was divided into three parts during which tutors could practice these steps with students: Part 1: Questions to determine individual student understanding and identify students who need support. Part 2: Team questions – taken from the Serway text “Questions to Improve Thinking” that structured the questions which were provided to tutors. Students worked in teams of three to four in all the tutorials for this second part. Part 3: Past examination questions. After the second OASIS assignment the percentage of non-participation rates in OASIS 1 and 2 for 2009 (11.9% and 16.6% respectively) were higher than in 2008 (8.6% and 12.3 % respectively) but comparable to 2007 non-participation rates. It was decided to re-develop the tutorial resources. The intention was to align the questions provided to students to those used in OASIS questions. The two lecturers were consulted about the OASIS questions to be used and questions from the text book that were similar in context to the OASIS questions were chosen. They were chosen to determine student understanding using the ‘Quick Quiz’ questions from the text as the Part 1 questions. The ‘Tutor Reply’ sheet was developed further to provide more detailed feedback about student responses to the tutorial questions. Tutorials 3 & 4 were both given a similar format, with a brief time for feedback about lecture style, then Part 2 provided some of the ‘Quick Quiz’ questions from the text and Part 3 provided selected text book questions that were aligned to the upcoming OASIS assignment questions. DATA COLLECTION The data for this project used the Mixed-Method approach (NSF Handbook, 2002) and provided information from interviews, focus groups, laboratory tutors, lab-tutorial tutors and surveys. All reporting of indicators is in terms of ‘positive-outcome’ strategies. Only those features revealed in the feedback are included in the remainder of this paper.


32

RESULTS THE TEACHING AND LEARNING PROGRAMME - FEEDBACK FROM TUTORIAL EXPERIENCES Tutorial 1 feedback shows that: tutors need time to become active tutors may prefer students to learn from text tutors need to prepare for each tutorial; hence material must be provided several days before. problems should be “easy to follow” laboratory-tutors need detailed answers some students can be reluctant to be involved in groups of 3 to 4 when they are in competition active tutors help students to learn using interactive reporting on white-board Tutorial 2 feedback identified several ideas and concepts that students did not understand. Feedback suggests fewer questions are required for tutorials than provided and that formula sheets would be useful. Tutors requested they be provided with answers to all questions before the tutorial by the course coordinator. Tutorial 3 feedback indicated: the need for several tutorial opportunities before students understand the dynamics and value of

their engagement that tutors become more active with experience as they “were active, revolving around the room to

provide assistance and check on student progress. After some time the room became much quieter and students settled to the work.”- report from observer

some students take at least two tutorial experiences before they become aware that the tutorial questions are from their textbook. Most had not read their text and had not attempted the ‘quick quiz’ or problem solving

additional tutorials were held before lectures and not in the usual tutorial time to review some topics – the Kirchhoff’s Loop laws tutorial was attended by 19 students.

Tutorial 4 focussed on several topics that needed emphasis and was used as the opportunity to survey student opinions about OASIS as an assessment tool. Students were equally divided about their preference and non-preference for OASIS as an assignment and assessment system. They liked OASIS since they can practice many times, at home or at university; it is more quantitative rather than qualitative so does not rely on writing skills. They disliked OASIS since it was possible to ‘fluke’ the answer rather than simply understand the Physics. When asked “Would OASIS assignments be better if they used more shorter questions and you were given

some questions that were similar, but not exactly the same, as the practice questions” students indicated they would rather stay with the current format and that they would be concerned about carry-on mistakes if the first part was incorrect. They preferred to have more questions on the same concepts instead of the same question with different values.

“If a different type of assignment was used what type might be best?” some students expressed a preference for OASIS, since it was able to be accessed from any computer, whilst others preferred a hard-copy take-home assignment that included many questions covering the whole of the course

Most students seemed to prefer the current format but had significant criticisms. The survey shows that: most students did not start OASIS 2 immediately if assessment items in other subjects take

precedence most students are using their text book to solve the questions most students do not use the Help Room since Help Room tutors need to be more supportive while 22 students out of 41 indicated there is little impact of other work and assessments, 19

students express that there is a significant impact of other assessment work that prevents them doing the OASIS 2 assignment. 21 students of the 41 indicate that other assessments they had at this time prevent them doing as well as they would like to do

only 8 students indicated that personal aspects in their lives prevented effective involvement with this assessment

23 students; 18 students and 17 students indicated that the difficulty of the ideas in the lectures prevented them achieving as well as they would have liked to


33

26 students expressed that the lab tutorials had little or very little impact to support them solving OASIS problems. 21 students; 16 students and 11 students indicated that lab tutorials did have a significant impact to support them to answer OASIS 2 questions. Further alignment between tutorial activities and OASIS questions is required.

Of the 8 students who did not complete OASIS 2 and responded to the questionnaire, 5 indicated that they did not start the assignment. None of these 8 students used Help Room tutor support at any time during the course. Four students indicated they simultaneously had other assignments with two indicating at least four assignments at this time. Their inability to manage workload prevented them from completing OASIS 2. There were other personal aspects in their life also prevented them completing the assignment. They experienced difficulty engaging with ideas in lectures and this prevented them achieving as well as they would like. The students indicated the tutorials did not significantly help them to solve the OASIS questions. THE TEACHING AND LEARNING PROGRAMME – FEEDBACK FROM THE FINAL INTERVIEW Of the informants, three were females and eight males. Ten were in their first year of university study, while one was second year. All had come directly to university from secondary school. Nine students indicated OASIS assignments improved their understanding of Physics, or gave some better understanding; eight students indicated that study for tutorial-aligned OASIS provided at least some improvement of achievement in tests; and six students indicated that study of tutorial-aligned OASIS provided at least some improvement in exam results. When asked what aspects might be improved, students had a range of suggestions from the need for tutorials to provide more explanation of concepts and be more useful to their learning, rather than contributing just an attendance mark to assessment. They suggested that laboratories need to be more aligned with course work and lectures. Finally they requested change from multiple – choice tests as they do not provide feed-back about how to improve understanding, and the need to provide reviews. EFFECTIVE, OR NOT? STUDENT ACHIEVEMENT IN PHYSICS 150 PHYSICS OF TECHNOLOGY OASIS ASSIGNMENTS The percentage non-participation in assignments 3 & 4 (See Figure 2) decreased relative to the continuing increases in 2007 and 2008. There may have been a positive effect of alignment of tutorials with assignments. The most dramatic effect is the reduction in non-participation in the final examination, from around 14% to just below 10%. Failed students whose result was DNS/DNC (‘did not sit/did not complete’) the courses also reduced from 15.7% in 2007 and 17.6% in 2008 to 11.5% in 2009.

CONCLUSIONS The positive response to aligning tutorials to the on-line OASIS assignments seems to indicate that students felt supported and encouraged to make more effort to attempt the on-line OASIS assignments. The tutorial methodology provided more engagement and group-work opportunities in a learning situation in which they knew that the work they were doing was coordinated and meaningful relative to impending assessments. The tutorials provided guidance, re-engagement and stimulated student cognitive development of ideas and concepts directly relevant to assessment. They encouraged group work, allowed tutors to identify student difficulties and provide opportunities to remediate student understanding. Student-student and tutor-student feed-back opportunities were enabled and these reduced the emphasis on competition. The tutorials sought to recognise the potential of each student and enable improved achievement by each student. This project was part of a wider project seeking to identify barriers to student learning, and find specific strategies that are “innovative, challenging, responsive to the needs of diverse learners, and under-pinned by sound

disciplinary and pedagogical expertise. Founded on an in-depth understanding of a wide ranges of teaching, learning and assessment

methods, of practices which support student learning, and the ability to select and apply different teaching methods in appropriate contexts.” (University of Auckland, 2009).


34

ACKNOWLEDGEMENTS The research was supported by a University of Auckland Faculty of Science grant that supported Brett Armstrong’s position as an observer. I would like to thank the Faculty of Science and Brett Armstrong for their support in this project.

REFERENCES Black, P & Wiliam, D. (1998) Inside the Black Box: Raising Standards through Classroom Assessment Phi Delta Kappan, 80,

139-148, Retrieved June 24, 2010, from http://www.pdkintl.org/kappan/kbla9810.thm Hattie, J (2003)Formative and summative interpretations of assessment information. on-line:

http://www.education.auckland.ac.nz/webdav/site/education/shared/hattie/docs/formative-and-summative-assessment-%282003%29.pdf

O’Byrne, J & Thompson, R, (2005) The tutorial benefits of on-line assignments : Mastering Physics in first year physics classes . Symposium Presentation, UniServe Science Blended Learning Symposium Proceedings 2005.

Popham, James W (2008) Transformative Assessment, e-book available from http://www.diesel-ebooks.com/cgi-bin/item/1416607269/Transformative-Assessment-eBook.html

Serway (2008) Physics for Scientists and Engineers 7th Edition Brooks/Cole Publisher Smaill, C (2005) The implementation and evaluation of OASIS: A web-based learning and assessment tool for large classes.

University of Auckland on-line: http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=04117016 Young, Freedman and Ford, University Physics with Modern Physics with MasteringPhysics™ Prentice-Hall, Pearson

Education Westat, Joy. F (2002) The 2002 User Friendly Handbook for Project Evaluation. National Science Foundation on-line:

http://www.nsf.gov/pubs/2002/nsf02057/nsf02057.pdf Wren, D.G. (2008) Using formative assessment to increase learning.Research report from the Dept of Research, Evaluation

and Assessment, Virginia Beach City Public Schools, http://www.vbschools.com/accountability/research_briefs/ResearchBriefFormAssmtFinal.pdf

Gordon, Nicholas, Teachers’ Reflection on the Challenges of Teaching Mathematics Bridging Courses

35

TEACHERS’ REFLECTIONS ON THE CHALLENGES OF TEACHING MATHEMATICS BRIDGING COURSES Sue Gordon, Jackie Nicholas Presenting author: Jackie Nicholas ([email protected]) Mathematics Learning Centre, The University of Sydney, Sydney NSW 2006, Australia

KEYWORDS: mathematics, bridging courses, teaching, reflection, email interviews

ABSTRACT The past fifteen years has seen a drift away from mathematical options in year 12 in which higher-level mathematical skills are taught (Brown, 2009), and an increase in the number of students enrolling in mathematics bridging courses. Effective teaching and learning in mathematics bridging courses are more important than ever. In this paper we present empirical data collected through a series of email interviews on the perceived challenges of teaching or coordinating these courses and strategies some employed to meet these challenges. We engaged educators in structured reflections on their approaches to teaching and learning in mathematics bridging courses, and created opportunities for participants to explore and potentially improve their own practices. Our findings suggest directions for future research and opportunities for reflection and debate about pedagogy in this important, yet little researched, context. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 35-40, ISBN Number 978-0-9808597-0-6

INTRODUCTION Mathematics bridging courses have been a part of the tertiary preparation landscape for many years, yet data on teaching and learning in these courses are scant. Recent reviews (Galligan & Taylor, 2008) of the limited research into bridging mathematics in the Australasian region have indicated consistent areas of investigation. These include evaluation of specific courses, diagnostic tests and other ways of determining students’ needs and overcoming mathematics anxiety. In ongoing research we investigate teachers’ and students’ views and experiences of mathematics bridging courses using email interviews. In this paper we focus on findings from participating academics about the perceived challenges of teaching or coordinating these courses. Through engaging educators in structured reflections on their approaches to effective teaching in mathematics bridging courses, the study created opportunities for participants to explore and potentially improve their own practices. Hodkinson (2005, pp. 117-118) postulates that although “our broad conceptualizations of learning can be fairly general, understanding how these conceptualizations can be applied in practice requires attention to the specifics of each location”. Our aims are to alert educators to the diversity of views about mathematics bridging courses and to stimulate reflection and active debate about pedagogy in this particular and important context. MacGillivray (2009) defines mathematics bridging programs as any preparatory program that enables a prospective student to obtain prerequisite or assumed knowledge in mathematics before commencing their degree program. We restrict this to preparatory courses that are intensive, 40 hours or less of instruction (in late January and/or February). At The University of Sydney the number of students enrolling in mathematics bridging courses has increased by 53% from 2001 to 2010. This reflects a national trend away from mathematical options in year 12 in which higher-level mathematical skills are taught (Brown, 2009). Brown (2009) quotes alarming statistics about subject choice at senior secondary; namely that the proportion of students in Year 12 whose highest level is Intermediate or Advanced mathematics has declined by 22% and 27% respectively from 1995 to 2007 (see Rubenstein, 2009, Appendix 1). Brown (2009) points out that Advanced mathematics is not esoteric pure mathematics, but mathematics widely regarded as necessary background for a first year Engineering student at university.


36

Concerns about the changes between learning mathematics at school and university have also been raised in the literature. Factors cited by Jennings (2009) that relate to transition issues in mathematics include: less prepared students, the fast pace of university mathematics courses, examination systems, the ever expanding curriculum and expected mathematical rigour. Leviatan (2008) refers to different ‘cultures’ in mathematics, with tertiary mathematics involving more abstract concepts and formal proofs. In NSW, seven universities run bridging courses in mathematics annually, an indication that these are considered a valuable resource for helping prepare students who have not studied the required level of mathematics prior to entering university. Hence information about teaching and learning in bridging courses is significant to the efforts to ameliorate our students’ difficulties with mathematics and help reduce attrition in first year for ‘at risk’ students.

METHODOLOGY The project draws on methodology developed by Gordon and colleagues (Gordon, Petocz & Reid, 2007) that collects data by means of asynchronous e-interviews with at most three returns. In these e-interviews the original set of questions are open-ended and designed to be specific to the respective respondent group, with the second and third interviews tailored to probe, in depth, participants’ responses from earlier rounds. In this project, teachers (and/or coordinators) of mathematics bridging courses at universities in NSW and students studying a mathematics bridging courses at The University of Sydney in 2010 were invited to take part in the qualitative research. Ten teachers and 15 students completed email interviews before our cut-off date. In the case of teachers, 7 completed the full round of at least two interviews and the remaining 3 responded to one interview. All participants gave informed consent and excerpts from transcripts are quoted in this paper under pseudonyms chosen by the participants themselves. In this paper we focus on specific aspects of teachers’ responses. Further work is in progress analysing students’ expectations and experiences of their mathematics bridging courses. The first email included a welcome message and a set of initial questions. Question 1 asked for background information such as teaching experience including teaching or coordinating mathematics bridging courses in 2010 or in the past. The remaining questions were deliberately open ended with the intention of enabling teachers to explore and articulate accounts of their own practice. These questions included the following. What are the most important things you expect students to achieve by studying a mathematics bridging course? What is a good mathematics bridging course? What teaching approaches or methods do you use, which are particularly helpful? What makes a good teacher in mathematics bridging courses? What are the most important challenges you have encountered in teaching and/or developing a mathematics bridging course? Follow-up email interviews probed and asked for clarification and amplification of respondents’ initial answers. Some examples are: Are there any teaching strategies that are particularly important for maths bridging courses over and above generic qualities you have described? What advice would you give to a colleague about to teach a maths bridging course for the first time? In previous projects using this methodology we have found that responding to our questions prompted participants to articulate their reflections with some care. In this way participants engage overtly with ideas about their own teaching that may have been in the background of their minds. As a participant of a previous study commented about the process of taking part in the email interviews: “such activities act as a source of enthusiasm and motivation/inspiration juice”. In this paper we focus primarily on teachers’ and coordinators’ responses to the following question and follow-up discussions: What are the most important challenges you have encountered in teaching and/or developing a mathematics bridging course? However, each transcript was considered in full by the researchers. Interpretations were prepared independently and then discussed by the researchers to ensure accurate accounts of teachers’ perceptions.


37

FINDINGS: CHALLENGES OF TEACHING IN A MATHEMATICS BRIDGING COURSE A number of themes emerged in response to our questions. While most educators referred to several of these themes in their e-interviews, we have separated the themes analytically and illustrate them in this section with short excerpts from the transcripts in italics.

1. CHALLENGE OF TEACHING A DIVERSE STUDENT GROUP IN A BRIDGING COURSE FORMAT Almost all teachers expressed views indicating that the distinctive environment of a mathematics bridging course presented singular challenges for developing appropriate teaching approaches. They discussed the challenges they faced in teaching a group of students with diverse mathematical backgrounds in such a short time frame. Hamlet spoke for many when he summed up: I suppose the key strategy particularly important for teaching a maths bridging course is being able to teach complex and new ideas to students who are weak in the subject in a short, compact time. And indeed this is the most difficult task in teaching! Jancsi commented that the diversity in the students’ backgrounds made it difficult for any one strategy to work: Sometimes you need to go beyond the course because you have all good students who are not challenged by the material, other times you have to spend a lot of time going over really basic examples on the board. Adam concurred with this when he said: A major challenge for me, then, is to pitch lessons at a level which won’t be too slow to cover the material, or to leave the advanced students merely revising topics they know reasonably well, but will not leave the less-prepared students behind. A coordinator of mathematics bridging courses, Hypatia2000, endorsed these views: that students arrive with a very wide range of backgrounds and that a good bridging course is: one that is at least twice as long as ours. Hypatia2000 found the most challenging students were: Students with even less than year 10 skills. Students who have not taken ANY maths since year 10 at school. Hamlet agreed that: there seems to be a significantly growing number of students appearing that do not even have a year 7-10 basic arithmetic/algebra skills. Like Hypatia2000, Selena thought the task would be hard if the class had many students with weaker mathematical skills. In her bridging class, Selena found that: [a] small number of students required greater assistance and this can mean that much of my time is dedicated to a small group. This was okay as students also help each other out and it was a small number. These excerpts revealed that developing appropriate teaching approaches for the wide range of student abilities in their classes was seen as a priority, and that the demands for skilful teaching were exacerbated by the short time frame of a bridging course.

2. CHALLENGE OF TEACHING COMPLEX MATHEMATICAL CONCEPTS Teachers’ responses indicated that they spent considerable time and effort finding ways to teach specific mathematical concepts. This theme overlaps with the first but with the focus shifted to the mathematics itself. Hamlet again spoke for many when he reported: In my teaching, I often spend time (often too much!) thinking of the optimal way of teaching a concept in a given time period. He believed that thorough and thoughtful preparation was essential so that: when a topic is taught, the delivery should be short and concise, yet so elegant that it flows like a coherent story and is easy to follow. Adam thought that an important strategy to meet this challenge was to simplify things if excluding the technical details made it: easier to understand the important ideas, at least at first. E found a particularly helpful strategy was focusing on the big picture first so that students get a grasp of that: before attempting to be technically precise or accurate. E recognised, too, that the symbolic language of mathematics could act as a barrier to students’ understandings, saying that: students can often find a lot of difficultly in understanding and expressing themselves in the appropriate syntax.


38

An added challenge articulated by Piper was: trying to get students to unlearn false mathematical notions and relearn correct mathematics. Selena mentioned that a common algebraic mistake included: the rearrangement of the equation to make a certain variable the subject. (e.g. 3x+4=0 => 3x=4). Students seem to forget this from time to time until I point it out. Sometimes the teachers were surprised at the level of students’ misconceptions or lack of knowledge about seemingly very basic mathematics. Selena observed: in my first lesson I had to explain that 3/3=1 and not 0 and tried to reinforce the concept so that the students will not make the same mistake by explaining that sharing 3 pizzas among 3 people would mean 1 for each person. The teachers stressed the importance of not assuming the students know too much, which for E meant that the teacher must be careful and: not assume ‘obvious’ steps. Selena gave the following example: Also when I ask some students to find the y-coordinate of a stationary point after they have the x-coordinate, some do not know which function (y=f(x), f'(x) or f''(x)) to substitute the x into. The above problem appears trivial so sometimes it is hard to anticipate the problems they would face.

3. CHANGING STUDENTS’ PERCEPTIONS ABOUT MATHEMATICS AND THEMSELVES AS LEARNERS Teachers were in unison that helping students change their perceptions of themselves as learners of mathematics was another major challenge. E thought that: the hardest challenge teaching wise is in trying to undo twelve years of self reinforcement that a person is inherently ‘not good’ at mathematics within a short time frame. Piper’s approach to this challenge was: to give the student a feeling of being in control over their studies and equipping them with the strategies and tools to ‘get stuck in’, for as Coolamon remarked: maths is all about doing. As an empathetic teacher Piper tried to: walk the very fine line between trying to break through student’s maths phobias and not coming across as patronizing or trying to trivialize the student’s past poor maths experiences. Adam calculated that this was an issue for between a third and half of his class and pointed out that if students did not have a ‘can do’ mind set: the rest is rather pointless. Always Hopeful reported on another aspect of students’ perceptions and experiences that teachers in mathematics bridging courses needed to overcome. This was the perception by students that: they do not need mathematics to study their program and therefore have a resentment of mathematics. If students were set in their view that maths is too hard and useless to their degree, proposed Jancsi, then there was little to be done to facilitate student engagement with the material. He saw that there was a wider societal arena to consider and observed that: essentially we are not a society that prizes education or learning. At the end of the day maths will not be relevant to most people. To change that perception would require a huge change in our society and economy. One of the possible reasons for the existence of bridging courses, conjectured Hypatia2000, was: – many students have not been challenged or stretched at school and expect there to be an easier path. Attitudes (only of some students) that if something is difficult you just drop back to an easier level - instead of working harder to learn something new that is initially confusing. This might help explain Jancsi’s comment that working outside of the course seemed to be: a big mental jump for students. He believed that students: do not seem to truly understand that the course requires work outside of contact hours. E felt it was important to be mindful of the fact that a lot of students had just finished high school, and were not as independent or mature as students just finishing first year second semester: It may be easy to trivialise this difference, but I think it is important to keep in mind. Chris rated the most important things he expected students to achieve by studying a mathematics bridging course as these: An increase in mathematical understanding A change in attitude towards mathematics An increase in confidence in doing mathematics Our three themes above summarise the challenges inherent in meeting each of these expectations.


39

4. ORGANISATIONAL AND LOGISTIC CHALLENGES We turn now to respondents, whose role included coordinating mathematics bridging courses. Coordinators reported that staffing issues were a major challenge for them as Hypatia2000 outlined. Some challenges were: finding staff who will teach creatively; finding staff who have enough sound mathematical knowledge to explain concepts and not just teach maths as exercises; finding staff who know where the maths is used in a variety of uni subjects from engineering to economics. Hypatia2000 took into account students’ needs, saying: as a co-ordinator I like the way we run things with a set amount to be covered in each daily session. This gives maximum flexibility to students who want to change classes for some reason or another. She added another challenge for those developing the bridging courses: finding suitable online resources for students to test themselves and get instant feedback. Other logistic and organisational challenges were described by the aptly named Always Hopeful. These included budgetary constraints, developing a format that fits the timeframe available and promoting the program to new students. In short: providing different levels and areas of bridging mathematics in the timeframe and be able to staff these. Chris narrated his story of initiating and co-developing statistics bridging courses some years ago. I drew on people who were more skilled and qualified in teaching stats than me to be co-developers of the materials for the bridging course. This was helpful as they knew the problems students face and could develop well-constructed worksheets and practical teaching sequences. We have reported the challenges experienced by teachers and their reflections on how they tried to meet these challenges.

DISCUSSION It is clear from the documented trends (Brown, 2009) in students' choices of mathematics at the senior secondary school level in Australia and studies of issues regarding students’ transition to university mathematics (Jennings, 2009) that the need for mathematics bridging courses at university is increasing and will continue to do so in the foreseeable future. Effective teaching and learning in mathematics bridging courses are arguably now more important than ever. Our paper provides preliminary information from one group of stake-holders in these courses, the teachers, by asking them directly about their views and concerns teaching mathematics in this singular environment. Participants identified challenges that are central to mathematics education in this specific and important context. Student diversity in mathematics background, motivation, conceptions of mathematics and approaches to learning mathematics are common challenges in all junior mathematics subjects at university, but the short time frame and expectations of students as to what can be achieved in a mathematics bridging course arguably increase the intensity of these challenges in the bridging course context. One area that warrants overt focus by teachers and developers of mathematics bridging courses is student engagement in the courses. As Coolamon points out: ‘maths is all about doing’ and without this, as Adam reminds us, 'the rest is pointless'. Our findings indicate that student engagement in mathematics bridging courses may be an issue for a relatively large group of students. Indeed questions have been asked (Wood, 2001) about the effectiveness of bridging courses in helping very weak students cope with first year mathematics. This suggests a clear need for research monitoring the progress of students who participate in mathematics bridging courses. Our research has implications for teaching. Data about experiences of mathematics bridging courses could inform advice given by teachers and careers advisors at secondary school and decision-making by students; perhaps discouraging at least some students at school from taking the easier level of mathematics when the going gets tough, as Hypatia2000 reported. The accounts of our participants and their willingness to engage in reflection on their practice indicate these teachers' commitment to improve teaching. As one respondent commented on our project: It is a busy time but it is also a good time to ask these questions as we have just finished our program and are reflecting on some of these points. Further, our findings provide a basis for reflection that goes beyond a focus on teachers' actions and approaches in mathematics bridging courses. At university, the mathematical concepts themselves — a focus on disciplinary knowledge — will always be a


40

priority of effective teaching. We argue that reflection and discussion is needed on teaching concepts at the appropriate levels and whether some simplification, as argued by Adam, is sometimes necessary even at the expense of rigour. A question raised by our research is: Are there effective teaching approaches for bridging mathematics that are distinct from those used in junior mathematics courses? This in turn relates to a question raised by Taylor and Galligan (2006) who ask: What constitutes success for students in mathematics bridging courses? A focus on the goals and constraints of mathematics bridging courses, the logistics of running courses within a budget and surrounding issues like staff flexibility, resources and time management, are areas where experienced coordinators can offer suggestions to individual teachers as well as others starting to develop mathematics bridging courses. Respondents’ ideas in this project suggest that debate could be fruitful as to what skills and strategies are important for bridging teachers and what proportion of time could be devoted to student led activities or group problem solving. Finally, we suggest that forums for teachers of mathematics bridging courses to interact informally and exchange ideas as well as formal pedagogical support and training are two ways of increasing a reflexive approach to teaching mathematics bridging courses that could benefit both teachers and students. Chris articulated a common problem. The challenge these days is I have little down time to do improvements as I am so heavily engaged both during semester and semester breaks in teaching. We hope to provoke a conversation about just that.

ACKNOWLEDGEMENTS We gratefully acknowledge the time and insights of the teachers and coordinators who participated in this research. The research was funded by a Small TIES Grant from The University of Sydney.

REFERENCES Brown, G. (2009). Review of Education in Mathematics, Data Science and Quantitative Disciplines. Report retrieved from http://www.go8.edu.au/go8media/go8-media-releases/2010/184-group-of-eight-releases-maths-review Galligan, L. & Taylor, J.A. (2008). Adults Returning to Study Mathematics. In H. Forgasz, A. Barkatsas, A. Bishop, B, Clarke, S.

Keast, W. Seah & P. Sullivan (Eds.), Research in Mathematics Education in Australasia 2004-2007 (pp. 99-118). Rotterdam: Sense Publishers.

Gordon, S., Petocz, P. & Reid, A. (2007). Teachers’ conceptions of teaching service statistics courses. International Journal for the Scholarship of Teaching & Learning, 1(1). Retrieved from: http://academics.georgiasouthern.edu/ijsotl/v1n1/gordon_et_al/index.htm

Hodkinson, P. (2005) Learning as cultural and relational: Moving past some troubling dualisms. Cambridge Journal of Education, 35(1), 107-119.

Jennings, M. (2009). Issues in bridging between senior secondary and first year university mathematics. In: R. Hunter, B. Bicknell and T. Burgess (Eds.), Proceedings of the 32nd Annual Conference of the Mathematics Education Research Group of Australasia. MERGA32 (pp. 273-280). Wellington, New Zealand: MERGA.

Leviatan, T. (2008). Bridging a cultural gap. Mathematics Education Research Journal, 20 (2), 105-116. MacGillivray, H. (2009). Learning support and students studying mathematics and statistics. International Journal of

Mathematical Education in Science and Technology, 40(4), 455 – 472. Rubinstein, H. (2009). A National Strategy for Mathematical Sciences in Australia. Strategy document written in consultation

with the Australian Council of Heads of Mathematical Sciences. Retrieved from http://www.amsi.org.au/pdfs/National_Maths_Strategy.pdf

Taylor, J. A., & Galligan, L. (2006). Research into research on adults in bridging mathematics: The past, the present and the future. In M. Horne & B. Marr (Eds.), Proceedings of the 12th International Conference of Adults Learning Mathematics, ALM (pp. 11-19). Melbourne: ACU National.

Wood, L. (2001). The secondary-tertiary interface. In D. Holton, (Ed.), The Teaching and Learning of Mathematics at University Level: An ICME Study (pp. 87-98). Dordecht: Kluwer Academic Publishers.

Holcombe, Pashler, Online Evidence Charts to help Students Systematically Evaluate Theories and Evidence

41

ONLINE EVIDENCE CHARTS TO HELP STUDENTS SYSTEMATICALLY EVALUATE THEORIES AND EVIDENCE Alex O. Holcombea, Hal Pashlerb Presenting author: Alex Holcombe ([email protected]) aSchool of Psychology, The University of Sydney, Sydney NSW 2006, Australia

bDepartment of Psychology 0109 University of California, San Diego,La Jolla, CA 92093, USA

KEYWORDS: inquiry, online, formative assessment, hypothesis-testing, science

ABSTRACT To achieve intellectual autonomy, university students should learn how to critically evaluate hypotheses and theories using evidence from the research literature. Typically this occurs in the context of writing an essay, or in planning the introduction and conclusion sections of a laboratory project. A student should distill relevant evidence from the research literature, evaluate evidence quality, and evaluate hypotheses or theories in light of the evidence. To help students achieve these goals, we have created a web-based “evidence-charting” tool (available at http://www.evidencechart.org). The main feature of the website is an interactive chart, providing students a structure to list evidence (usually drawn from research articles or experiments), list the theories, and enter their evaluation of how the evidence supports or undermines each theory/hypothesis The chart also elicits from students their reasoning about why the evidence supports or undermines each hypothesis, and invites them to consider how someone with an opposing view might respond. The online chart provides sortable summary views so that one can, for instance, see the evidence indicated to be most important for each hypothesis. Upon completing a chart, the student is well positioned to write their essay or report, and the instructor can quickly provide formative feedback indicating whether the student has successfully reviewed the literature and understands the evidence and theories. These benefits are being evaluated in the context of introductory and advanced psychology classes. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, pages 41-46, ISBN Number 978-0-9808597-0-6

INTRODUCTION University graduates should be independent thinkers. In today’s world, the knowledge needed to succeed in many occupations can change rapidly. Specific content information learned at university frequently becomes outdated or obsolete after a few years (Scardamalia & Bereiter, 2003). Today, a wealth of task-relevant information is often available at one’s fingertips through the internet. However, assessing which information is truly relevant to the task question at hand can be difficult. Once the relevant information has been identified, the next step can be even more difficult. The inquiring person should next critically evaluate the information and synthesise it into an overall answer. Consider an IT manager contemplating which of three types of computers would perform better for a certain purpose. Or, a veterinarian trying to decide which of four possible treatments to administer to a horse with a particular disease. Or a business consultant facing a series of deadlines who wants to know whether drinking coffee or taking naps would be better for his productivity. For each of these questions, there may be no authoritative reference work available that provides the answer. To make an intelligent decision, these professionals must consider what kind of evidence would be relevant to their decision, how they might acquire that evidence, seek it out, organise it, and synthesise it into an overall answer. These skills of independent inquiry do arise in many university curricula. More precisely, a need for these skills sometimes arises, although the skills themselves are not always taught effectively. The skills are utilised in essay assignments or laboratory research projects. For example, for essays in certain science classes students must examine the research literature to evaluate theories or hypotheses. Laboratory projects also have potential for fostering intellectual autonomy. In a basic laboratory exercise, students are given a set experiment and learning is restricted to understanding a


42

specific experiment and the analysis of its results. However, in cases that foster more intellectual autonomy, students are asked to write an introduction that sets out hypotheses or theories, and in the conclusion evaluate the theories in light of their own results and that of results reported in the literature. For both a research-based essay assignment and a lab report that engages with the research literature, a student may need to perform the following steps: 1. In response to a question or point of contention, formulate candidate theories or hypotheses 2. Glean relevant evidence from original data or from scientific literature 3. Organise the evidence and evaluate how each speaks to the theories or hypotheses considered 4. Synthesise the evidence and their interpretation of it into an overall answer. In the context of a laboratory report or scientific essay, students should already be performing each of these tasks. It is our experience, however, that students frequently fail to successfully complete one or more of these tasks. Unfortunately, identifying where the failure occurred can be difficult. Assessments of student work frequently consider only the final product of the process—a finished report or essay. This makes it difficult to determine which steps of the process were done properly and which were not. The difficulty is compounded by the fact that many students do not write clearly. While helping students clarify their writing can sometimes be done with solely the final document, identifying which of the preceding steps went wrong is more problematic. And without focused feedback regarding which steps were not performed properly, many students will persist in their mistakes. The “evidence-charting” tool described below is designed to achieve two outcomes: 1. Support student performance of the four steps identified above. 2. Create evidence of student performance of these steps, and make it easy for an instructor to

assess. The evidence-charting tool we have created is embodied in a website. The tool is viewable at http://www.evidencechart.org, and hereafter this particular software will be referred to as EvidenceChart. It provides a structure with slots in which the student adds information to create an organised summary of their research and some of their thinking. As the student proceeds, the constant presence of the structure reminds the student of what is to be done.

CHARTING THE EVIDENCE The evidence chart is oriented towards answering an empirical question. It revolves around the candidate hypotheses, relevant evidence, and how each piece of evidence speaks to the hypotheses. The EvidenceChart site has slots for this information in its two-dimensional tabular structure. Each column addresses a particular hypothesis, and each row a particular piece of evidence. Each interior cell of the resulting matrix is the meeting point of a theory with a piece of evidence. This tabular representation is rather intuitive and has apparently been invented repeatedly over the years. It has been used systematically in communities of intelligence or national security analysts, where it is called the “Analysis of Competing Hypotheses” method (Horn, 1999). It has also been used in classroom settings, but reports on its usage are scant. The exception we have found is the Belvedere education project, which includes evidence charts in its Java software for student collaborative inquiry, wherein students created hypotheses, discussed them, and made diagrams as well as an evidence chart to further their inquiry (e.g. Suthers, Toth & Weiner, 1997). The software does not support online collaboration, but is still available as functioning Java software from the project website. Our effort has been restricted to making a website with easy-to-use evidence charting, plus accessory functionality that assists instructor evaluation and response to what the student has done. By creating a website focused on this relatively narrow enterprise, we hope to keep the programming challenge manageable and maintainable while still having enough functionality for the site to be useful in various contexts.


43

Figure 1: An empty evidence chart immediately after its initial creation and assignment of a title. From www.evidencechart.org Our approach is design-based research: implementing and improving our evidence-charting tool in iterative fashion. Following the use of the tool in a university class, we collect feedback from students and instructors and then revise the website and associated instructional material and assessments for the following semester.

Figure 2: A portion of an evidence chart. The chart was created by Denise J. Cai (UCLA Physiology) and is used with her permission. Evidence (rows, labeled in the leftmost column) and hypotheses (column headers) have been entered, and the degree to which each piece of evidence supports or undermines each theory has been indicated. The student should continue by entering text at each interior cell of the matrix to indicate why the corresponding evidence supports or undermines the corresponding hypothesis. A further aspect is a ‘contrarian view’ of each cell, in which the student is encouraged to play devil’s advocate and describe the best argument against the position they have taken in this dominant view. In the current iteration, when the student visits http://www.evidencechart.org, they begin with an empty evidence chart, as shown in Figure 1. The underlined links shown in the screenshot (Figure 1) indicate to the student that she should add hypotheses and evidence by clicking on the indicated text, after which text input boxes appear and prompt the student to enter corresponding information. As a student does the work outlined in the four steps described in the introduction, she gradually populates the chart. A portion of one such chart is pictured in Figure 2.


44

In this chart, each row represents a published scientific article or monograph with results that bear on the question of how sleep affects memory consolidation. Each column describes a different hypothesis regarding the role of sleep in memory consolidation. At the intersection of each row and column, the student should: Rate the implication of the evidence for the hypothesis, on a scale spanning “strongly undermines”

(colour-coded with red) to “strongly supports” (green) the hypothesis. Enter a phrase explaining why they believe the evidence supports/undermines the theory. This is

termed the “dominant view”. Engage in “devil’s advocate” thinking by entering a phrase defending the view opposite to what

they have indicated in the dominant view. This in entered in an area revealed by clicking on the View menu.

These three functions occur at each cell of the table and systematically coax the student to think critically about the evidence and the hypotheses. The text entered into the contrarian view encourages the student to take another perspective, allowing the student herself to provide the useful and classic ‘devil’s advocate’. The ‘devil’s advocate’ technique descends from the classic method of Socrates and is commonly used in educational contexts such as law schools; the law professor customarily challenges a student’s argument by raising arguments against the student’s position. Without some kind of prompting, many students writing an essay or lab report will amass arguments for their position but never think actively about the best arguments against their position. The evidence chart encourages contrarian consideration without the requirement for active intervention by an instructor. The student’s activity described so far is primarily analytic, considering each piece of evidence as an individual. Eventually, the student should shift to synthesizing the evidence and its implications to arrive at a coherent view. Such synthesis of possibly disparate and contradictory pieces of evidence is clearly a subtle enterprise that cannot be reduced to a formula or algorithm. It requires more than simply ‘adding up’ evidence that seems to be for and against an argument. The evidence chart web application does however provide a small degree of assistance. By clicking in a drop-down menu associated with each column, the student can sort the rows by degree to which he has indicated the evidence supports or undermines the theory. This can be very useful for considering the strongest evidence for or against a hypothesis—particularly for larger charts, such as the full chart excerpted in Figure 2, which contains 20 rows in its full form. A further feature, not yet implemented, may sort the evidence rows by the extent that they discriminate among all the theories.

USING EvidenceChart TO IMPROVE FEEDBACK AND ASSESSMENT When a student receives a poor grade or mark, the student should be told which aspects of their performance were responsible for the poor outcome. Lab reports and research essays can include several steps before the writing begins and from a poor final product, it can be difficult to judge which steps were at fault. Some students are on the wrong track well before beginning to write, but persist in following their ill-conceived notions or process to a mistaken conclusion. The EvidenceChart webtool makes it easy for an instructor to assess student performance of the suggested steps prior to the writing of a final report or essay. Through the website, user accounts of students in a particular class are grouped together, and class instructors can view the evidence charts they create as part of the class. By requiring each student to prepare an evidence chart, instructors can assess whether a student has found the appropriate related evidence, been able to articulate competing hypotheses, and has some understanding of how the evidence supports or undermines each hypothesis. Thanks to the succinctness of evidence charts, they can do so quickly. Without such a concise format, in large classes it is often impractical to provide individual attention to students prior to final assessment. A particular advantage of the web-based implementation of evidence charting is that instructors can “drop in” without the student needing to submit anything formal. In the evidence-charting site, the instructor can add a note for the student indicating which parts appear to be a problem. For a project or essay, rather than have a single deadline corresponding to the final product, students can be required to complete an evidence chart online some weeks before the essay or product is due. As a graded component, the instructor may simply wish to confirm that the student has done something substantial, but for formative assessment can take the opportunity to guide the student with comments on the chart. In addition to correcting students who misunderstand the hypotheses or


45

related evidence, this also curbs the student procrastination problem by making it a requirement that students do some substantial research and thinking well before the final assignment is due.

CURRENT EXPERIENCE AND PROSPECTS Creating the web application has been a large software development effort, involving many cycles of planning, programming, and assessing the utility and usability of the website. To be truly successful the tool must be very easy and quick to use, or students will resist it. This provides a significant user interface and web programming challenge. As the website has not been stable but rather changed and improved continuously, with intermittent bugs arising in the process, we have not yet mandated that students use it in any class. However, for two semesters the site has been presented to the students as a tool that could benefit them and which they may use if they wish. We have also used it in our own unrelated scientific research to evaluate the viability of various scientific hypotheses. In using the tool ourselves for professional scientific research, we have been surprised by its effectiveness at eliciting new critical insights. For example, one of us studied a particular visual illusion for two years and formed various opinions of the theories that have been proposed to explain the illusion. Simply to test out the website functionality and ease of use, it was decided that an evidence chart regarding the topic would be constructed. The process prompted focused consideration of how each piece of evidence could or could not be reconciled with each theory. This proved very productive, as several novel insights were gained. Although previously much of the evidence had been considered extensively in light of one or two theories, never had each piece of evidence been considered for each. We believe that most students as well as working scientists also do not usually approach a problem very systematically. Many scientists know that there is nothing like writing an article or grant to force oneself to consider a theory more carefully. However, writing prose can be daunting and considerations of exposition, clarity, and organisation can become prominent before one gets through very much evidence. In contrast, the very limited space provided in the cells of an evidence chart elicits a short phrase or two accompanied by careful thinking. The blank space of those entries where the evidence has not been fully evaluated are persistent reminders that one has been negligent. The existence of such omissions are easily forgotten or never even realized without an evidence chart. Furthermore, the result of the process provides a product that facilitates synthesis of the evidence. In traditional prose format, synthesis seems more difficult. One reason is undoubtedly the limitations of working memory: it is simply hard to keep in mind the points made in many different paragraphs regarding how a half dozen pieces of evidence relate to three different theories. Student feedback on the usefulness of the tool has been limited to date, but encouraging. At the University of Sydney, the tool has been presented to students in a large introductory psychology class consisting mostly of first-year students, to fourth-year (honours year) students working on a year-long research project, and to a few postgraduate students in psychology. Evidence charting was described as entirely optional and it seems that only a small proportion chose to attempt this additional activity. Feedback has been solicited via prominent hyperlinks on the website and electronic surveys emailed to many of the students. Responses have been few, limited to a dozen or two, and have consisted of two types. First are reports of problems or perceived problems with the functionality of the website. For each of these negative reports, together with our programmer we have been able to quickly resolve the issue. All other comments have been positive and have often been provided by a person who also complained about a possible bug with the site. When receiving a complaint, we take that opportunity to engage the person and ask about the site’s general utility. Here are a few of the comments we have received: Because I have reasonably slow internet, occasionally the program had trouble saving the information I had just added. Which was mildly annoying, but overall it was a really awesome tool. I'll definitely use it again when I restart my degree in a few years. =) I created an account and successfully started using EvidenceChart - it is seriously amazingly helpful because Microsoft Word and Excel are absolutely crap for this sort of thing..... And like I said, this is amazingly helpful in sorting out the literature! Thanks for getting this out to us :) A PhD student who we commissioned to test the site by making a chart associated with her doctoral work provided the following feedback: It makes me think of the contrarian view, which is great! While I think about this all the time, it's actually really helpful to verbally articulate it and then document it! It's also been helpful in dissociating between the strength of confirmation/opposition for a theory vs rigorous/"well-doneness" of a study, as


46

mentioned before. I'm sure it'll help me gain more "ah-ha" moments as I start working on a less familiar topic. This doctoral student, together with others, mentioned the difficulty of choosing the best level of granularity for the evidence. In the case of preparing an evidence chart for a scientific essay comparing theories, for example, should each row refer to an individual experiment, an entire scientific article, or a set of scientific articles containing similar experiments? This can be difficult to know before most of an evidence chart has been constructed. When the appropriate level of granularity has been chosen, certain pieces of evidence may be highly related; for example, they may all bear on a single aspect of a hypothesis. Ideally, this evidence should be grouped together or perhaps be part of a larger hierarchy. However, it has been difficult to envisage software support for this without making the user interface substantially more complicated. Our aim is to keep to a simple design that a novice can use immediately after nothing more than a one or two-minute explanation. As the software has been tested by many dozens of student volunteers and crashes and bugs are now rarely if ever encountered, we are ready to move to the next phase of the project: mandating that students create an evidence chart prior to writing their essay or lab report, and providing them with rapid formative feedback on the basis of the chart. Following this, there are plans to modify the software to allow for collaborative group editing of evidence charts. This will allow groups of students to work together on the chart (using their individual logins), allowing them to learn from each other, even across large distances, and more independently from the instructor. Full-formed prose writing is clearly not an optimal format to start with when planning a critical essay. It is not surprising, then, that long before evidence charts and computers were invented, there were other techniques that students used to plan their essays. For example, many scholars and students, especially in the humanities, put bits of information on individual small cards or “index cards”. Typically, one piece of evidence is written on each card, similar to the individual rows of an evidence chart. After the evidence is amassed, the cards are assembled into a linear or two-dimensional array that has some sort of correspondence with the argument or composition being planned. The potential to create practically any structure with this technique means it is suited to any purpose. At the same time, however, it does not provide a guiding structure for a student who is not yet a master of the process. Similarly, concept-mapping and mind-mapping are very flexible but provide few relevant structural constraints. Argument maps are highly structured and very promising for concisely representing arguments but require extensive training to learn (van Gelder, 2002). An intermediate between these extremes, something like evidence charts, may eventually take hold as a helpful tool for students and professionals. The added interactivity and limitless functionality possible in internet-connected software will undoubtedly be an intimate part. The evidence-charting tool is useful now and we hope it is moving in the right direction to help students and scholars work efficiently, systematically, and think critically.

ACKNOWLEDGMENTS This work was supported by the US National Science Foundation (Grant BCS-0720375 to H. Pashler, and Grant SBE-582 0542013 to the UCSD Temporal Dynamics of Learning Center) and by a collaborative activity grant from the James S. McDonnell Foundation.

REFERENCES Horn, R. (1999). Analysis of Competing Hypotheses. In Psychology of Intelligence Analysis. Center for the Study of Intelligence,

CIA. Scardamalia, M. and Bereiter, C. (2003) Knowledge Building. In Encyclopedia of Education, MacMillan. Suthers, D.D., Toth, E., & Weiner, A (1997). An Integrated Approach to Implementing Collaborative Inquiry in the Classroom. In

Computer Supported Collaborative Learning '97, December 1997, Toronto. van Gelder, T. J. (2002). Enhancing Deliberation Through Computer-Supported Argument Visualization. In P. Kirschner & S.

Buckingham Shum & C. Carr (Eds.), Visualizing Argumentation: Software Tools for Collaborative and Educational Sense-Making. London: Springer-Verlag, pp. 97-115.

Jacobson, Richards, Kennedy-Clark, Thompson, Taylor, Hu, Taylor, Kartiko, Scenario-based MUVE for Science Inquiry

47

SCENARIO-BASED MUVE FOR SCIENCE INQUIRY Michael J. Jacobsona, Debbie Richardsb, Shannon Kennedy-Clarka, Katherine Thompsona, Charlotte Taylorc, Chun Hua, Meredith Taylorb, Iwan Kartikob Presenting author: Michael J. Jacobson ([email protected]) aCentre for Research on Computer-supported Learning and Cognition (CoCo), University of Sydney, Sydney NSW 2006, Australia bDepartment of Computing, Macquarie University, Sydney NSW 2109, Australia cSchool of Biological Science, The University of Sydney, Sydney NSW 2006, Australia

KEYWORDS: science education, inquiry learning, virtual environments, collaborative learning, pedgagogy

ABSTRACT The development of scientific inquiry skills is a core element of the draft national science curriculum for Australian secondary school students. Yet, despite the prominence of inquiry learning in the curriculum, the use of integrated classroom technology to facilitate inquiry learning is difficult for teachers to implement successfully without support in Australian secondary schools. The purpose of the Virtual Worlds project, which commenced this year, is to conduct learning and cognitive sciences-based research into the potential of scenario-based Multi-User Virtual Environment systems to promote, and perhaps enhance, secondary school learning experiences. In this paper we consider a number of existing science education-based multi-user virtual environments and introduce our project including our goals, approach and scenario underpinning the virtual world. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 47-52, ISBN Number 978-0-9808597-0-6

INTRODUCTION The difficulties that students experience learning science, as well as the low overall interest in science that many students have, are well-documented (Barnett, Barab, & Hay, 2001; Bransford, Brown, & Cocking, 2000; de Jong & van Joolingen, 1998). There have been many arguments advanced about the potential serious consequences of a pervasive lack of understanding of important scientific perspectives both for individuals and for an informed citizenry that must increasingly deal with a variety of local and global challenges concerning interconnected social and physical systems (Jacobson & Wilensky, 2006; Rutherford & Ahlgren, 1990). Of course, no single panacea can possibly address the multitude of cognitive, pedagogical, and social factors that contribute to the decline of students’ interest in, and the poor overall level of understanding of science. However, several research studies suggest that great potential exists to help address issues, such as a general lassitude towards science, through the educational use of appropriately designed digital media sometimes referred to as a “3D” game or multi-user virtual environment (MUVE) that can run on currently available multimedia and Internet capable computers in schools and homes (Barab, Thomas, Dodge, Carteaux, & Tuzun, 2005; Barab, Warren & Ingram-Goble, 2006; Dede, Clarke, Ketelhut, Nelson & Bowman, 2005a). In fact, the 2009 Australia – New Zealand Horizon report (2009) indicates that virtual and alternate realities are one of the technologies to watch over the next five years in classroom education as they are proving to be an effective means of attracting and gaining students’ attention and interest. To ensure engagement, we are particularly interested in scenario-based MUVEs, such as Quest Atlantis (Barab, Warren & Ingram-Goble, 2006), River City (Dede, Clarke, Ketelhut, Nelson & Bowman, 2005a), and Virtual Singapura (Jacobson, June Lee, Hong Lim & Hua Low, 2008), that are underpinned by a scenario and are more akin to a role-playing game than a virtual lecture or meeting room. In this paper, after providing an overview of research on science learning in MUVEs, we introduce our project that is aimed at the investigation of centrally important issues related to learning in MUVEs, and to provide practical approaches for their integration into Australian science education classrooms.

OVERVIEW OF RESEARCH INTO LEARNING IN MUVES Educational researchers have recently argued that the affordances of highly interactive game-like systems are well suited to support many of the recommendations emerging from learning sciences and educational research related to how students learn (Barab, Thomas, Dodge, Carteaux, & Tuzun,


48

2005; Jacobson, June Lee, Hong Lim, & Hua Low, 2008; Jacobson, Kim, Miao, Shen, & Chavez, 2009; Ketelhut, Nelson, Clarke, & Dede, 2010; Squire, 2005; 2004). Indeed, the question seems to be not whether or not to use the technology in education, but rather what is the best way to use the technology in terms of effectiveness (Squire, 2005). Given the motivational and active learning potential of MUVEs or interactive game systems, researchers have begun to develop highly interactive 3D environments that are aligned with specific content in school curricula, rather than attempting to repurpose existing commercially available games. For example, Squire, Barnett, Grant and Higginbotham (2004) conducted a study in which 61 students used a game, Electromagnetism Supercharged!, to learn science content related to electrostatics, while 35 students in the comparison condition did not use the game. Students in the experimental condition scored significantly higher on conceptual items related to electrostatics on the posttest compared to the comparison condition. No significant gender differences were found. Other qualitative findings were that both boys and girls were initially eager to play the game, but that many boys lost interest by the second day once they felt they had “beat” the game. Interestingly, girls were less interested in “playing-the-game-for-points” mode and instead explored the game as a simulation in which they collaboratively worked to record their actions and to share their results with student peers. Quest Atlantis (QA), employs a scenario-based educational MUVE environment that allows students to travel in a virtual space to perform in-class or after school educational activities, talk with other students and mentors, and build virtual identities (Barab et al., 2005). Students have been found to be motivated when engaged in QA “quests” and to respond to the narrative aspects of the immersive experience more so than the “game-like” features (Barab et al., 2005). Also, a study of sixth grade students in the U.S. who used QA found statistically significant increases in the students’ conceptual knowledge of cells, while another study found students constructed solid understandings of ecological concepts such as erosion and eutrophication after using the QA Anytown unit (Barab et al., 2005). In a study of the River City MUVE involving the participation of approximately 700 students in grades five to eight in two different U.S. cities it was found that the experimental group students who used River City over a two-week period had significantly higher science content knowledge and science inquiry skills gains compared to the comparison condition students who used a paper-based version of the science inquiry curriculum. A large proportion of the students came from low SES communities, so another important finding in this study was that compared to the control group, students in the experimental group were highly engaged in their learning activities with the system, had improved attendance and less disruptive behaviour, and made significant learning gains. Overall, these findings, which have been substantiated by recent research (Ketelhut, 2010), suggest that the use of an engaging scenario via a multi-user virtual environment offers a learner-centred pedagogical approach that can enhance student academic achievement and may particularly help teachers to reach students struggling with motivation, self-worth, and lack of content knowledge. However, Dede et al. (2005a) also stress that important research is still needed in order to explore how students might better transfer or apply subject specific knowledge and skills learned in immersive virtual environments to new situations and contexts outside of the virtual.

RESEARCH FRAMEWORK, AIMS, QUESTIONS AND APPROACH Science is often presented to learners as a body of information and facts to be remembered; a strategy that provides learners with little opportunity to engage and process the information (Siorenta & Jimoyiannis, 2008). Science teachers acknowledge that there is a difference between ‘doing’ science and science education (la Velle, Wishart, McFarlane, Brawn, & John, 2007). The challenge, of course, is how to implement research-validated and authentic pedagogies for learning core subject knowledge and skills in ways that are logistically viable in formal and even informal learning environments. The use of technologies, such as MUVEs, in a science classroom may better foster scientific habits than traditional pen and paper activities as students have the opportunity to simulate science processes, such as data collection, without the real world messiness or risk of harm (Bainbridge, 2007; la Velle et al., 2007). The research methodological framework underlying this project is design research involving the use of two different educational MUVEs (described below) in secondary school classrooms and in teacher education courses. Design research involves conducting formative studies in real world contexts such


49

as classroom environments that test an innovative theory or research based educational design, and that in turn iteratively refines the design of the learning environment over time (Brown, 1992; Collins, Joseph & Bielaczyc, 2004). In terms of learning theory, the design of the technology and the curriculum are informed by general learning sciences theories such as situated cognition (Brown, Collins & Duguid, 1989) and distributed cognition (Salomon, 1993) as well as what might be called “focused” cognitive theories of conceptual change (Salomon, 1993), and knowledge transfer (Bransford & Schwartz, 1999; Gick & Holyoak, 1987). More recent cognitive and learning sciences theoretical perspectives that have important pedagogical implications, such as analogical encoding theory (Gentner, Loewenstein & Thompson, 2003) which suggests that analogies promote attention to commonalities between objects, activities and concepts and productive failure (Kapur, 2008) which suggests that using low structure in the initial phase of a problem can result in deeper learning, will also ground some of the specific experiments to be conducted and be directly relevant to the first two research questions given below. This project aims to conduct learning and cognitive sciences-based research into the potential of MUVE systems to promote and perhaps enhance secondary school learning experiences. To achieve these aims and based on issues identified in the literature, three central research questions related to the integration of immersive virtual environments in Australian classrooms are being investigated in this project: First, how might learners construct deep and transferable understandings of important and

challenging scientific knowledge and skills through specifically designed pedagogical experiences involving immersive virtual learning environments?

Second, how might science learning activities involving MUVEs motivate students both to learn science knowledge and skills as well as to develop positive attitudes and predispositions towards science?

Third, what roles will teachers assume when teaching with MUVEs and what types of professional development and support will they need in order to effectively integrate and use MUVE systems in their classrooms?

The approach to be used in this program of research is to conduct classroom-based, quasi-experimental research studies into the efficacy of multi-user virtual environments to help students learn content knowledge and skills in ways that are deep, adaptable, and transferable. The main research contexts for the project consist of Australian secondary school classrooms. The role of teachers is pivotal in the successful integration of technology into a classroom setting (Becta, 2004; Urhahne, Schanze, Bell, Mansfield, & Holmes, 2010; M. Webb & Cox, 2004; M. E. Webb, 2005). In this project science teachers from government and private secondary schools in New South Wales are currently collaborating with the research team on the design of the scenario, activities and assessments. Regular meetings with the science teachers have involved the training of staff in the use of Virtual Singapura (VS), the scenario-based MUVE we are utilizing in the first year of our project as discussed below, so that teachers are familiar with the type of technology that can be used in science learning and can provide input on how to design the materials that meet the needs of the students and address the outcomes of the curriculum. This project has also involved several trials with pre-service teachers both to pilot the materials as part of the ongoing cycle of iteration and to gain an understanding of user processes. We hypothesise that as a result of this level of teacher and state education level feedback and interactions that the participating teachers and schools will feel more “ownership” and long-term positive interest in the project’s virtual learning environments and innovative pedagogical approaches, and thus be more receptive to the professional development experiences that will be provided before the research implementations, in contrast to other research that has shown technological innovations in schools are often perceived as a distraction or imposition on the teachers who consequently resist initial and/or subsequent uses of these approaches (Henriques, 2002; Lee, 1997). We also hope that having an experienced cohort of teachers from the first year of the project will help with recruiting and training teachers to be brought into the project in other schools in the second and third years of the project.


50

MUVES AND LEARNING IN AUSTRALIAN SCHOOLS Currently available commercial 3D virtual environments are almost exclusively oriented to “gaming” experiences that students (and many adults) find exciting and challenging, but which unfortunately are not aligned with subject specific knowledge and skills that students must learn in formal school settings (Dede et al., 2005a; Kirriemuir & McFarlane, 2004; McFarlane, Sparrowhawk, & Heald, 2002). Consequently, in order to conduct research into learning with MUVEs in Australian schools, it is necessary at this time to utilise non-commercial educational MUVEs that have been developed as part of prior research as well as to develop a new Australian-oriented MUVE. This project is employing both approaches. The project is initially utilising VS, a scenario-based MUVE prototype built in ActiveWorlds1 which has already been successfully deployed in classroom settings in Singapore. VS provides an environment in which secondary students learn science inquiry skills such as proposing research questions, hypothesis formation, identification of dependent and independent variables, analysis of data, and interpretation of findings in relationship to hypothesised outcomes as well as learn about communicable diseases and the impact of humans on the environment. The scenario for VS is based on historical information about disease epidemics in 19th century Singapore, which are similar to those in Sydney during the same period. In this virtual world, 21st century students go back in time to help the 19th century Governor of Singapore, Sir Andrew Clarke, and the citizens of the city figure out what is causing various illnesses and to propose viable 19th century solutions to stop the epidemics. When students teleport back to 19th century Singapore, they arrive at the Boat Quay in 1874 and then use their avatars, which are computer generated characters on the screen that they control and communicate through, to explore portions of the historical city that include the Tan Tock Seng Hospital (Chinese Pauper Hospital), rickshaw tenement houses in Chinatown, houses in the wealthy European neighbourhood, and so on. As the 21st century student scientists investigate the causes of diseases that manifest during different seasons of the year, they meet computer-generated VS citizens (i.e., programmed to be adaptive with intelligent agent technology) and visit various locations in the city to view pictures (that include written descriptions and information relevant to the inquiry activities), inspect various digital objects to gather information, and obtain air, water, and bug samples at virtual data collection stations (see Figure 1). The students communicate with their team members (usually in groups of four) using a group-chat function and can also chat with the various 19th century agents they meet, such as the doctor and nurse in the hospital, coolies on the street, a scholar at the medical school, the poor mother of a sick child, and so on. There is also a VS Lab Book that currently introduces students to aspects of inquiry learning such as data collection, making observations, evaluating data and making recommendations. The VS MUVE was developed so that students in secondary classroom settings could easily use it, and it also has a number of research design features to collect specific types of data to investigate both the learning outcomes as well as learning processes or trajectories associated with the experiences that students have in the virtual environment. For example, log files are automatically collected as the students communicate with their team members using the group-chat function or when they interact with the various 19th century characters. Also, log files of all location information are collected for each of the students’ avatars and screen capture software with webcams and microphones are used to record exactly what is on the screens as students use VS as well as video and audio of any conversations they may have with fellow students, teachers, or members of the research team. This process data, in turn, is used to complement the analysis of pretest and posttest summative assessments related to the research questions regarding learning for deep understanding of scientific knowledge and skills and knowledge transfer in virtual environments. Results involving the use of VS by grade seven students in Singapore found that all of the students were highly motivated by the experience in the virtual environment (Jacobson et al., 2008).

1 http://www.activeworlds.com/


51

Figure 1: Virtual Singapura bug catcher Figure 2: Predator Model created in Unity3D We are currently developing a new virtual world with Australian-themed scenarios using the game development tool Unity3D2. The topics being considered include: Year 8) ecosystems; Year 9) diseases; and Year 10) evolution. These topics have been identified and suggested by the science teachers in our focus groups. While these scenarios are currently more focused in the biological sciences, we intend to develop modules for physics and chemistry content. To this end, as part of this Discovery project and specifically as the focus of a concurrently running Linkage Project with the Centre of Learning Innovation in the New South Wales Department of Education we are currently developing a number of simulation models that will be incorporated into the Australian MUVE. All three school years have common skills concerned with scientific inquiry: hypothesis, investigations, experimental design, the use of secondary resources and data, using equipment, managing risk, collecting data including ICT, analysis, conclusions and communications. The initial in-world activity is being developed for Year 8 science with additional activities being developed in years two and three of the project. The new virtual world is based on the national science curriculum that will be implemented into Australian schools in 2011. As in VS, RiverCity and QA, we have an overarching scenario, the students will work for the Intergalactic Environmental Preservation Association (IEPA). The students will be part of a team of interstellar scientific detectives working for the IEPA. As the team travels around the galaxy they are given certain problems to solve. For example, the team is asked to visit a small terrestrial planet where several species of mega-fauna, that once thrived, are disappearing. The team’s job is to discover what the possible reasons are for these disappearances. The students will be able to interact with a hunter, ecologist and climatologist that live on the planet. They will also be able to observe animals hunting, interact with virtual timelines and biological records and run population models. As an example, in Figure 2 we show a predator model involving grass, sheep and wolves which we have developed in Unity3D as a prototype to allow students to model the behaviours and relationships between these populations. There is a progression in concept development as students move from one problem to another. The initial problem will investigate the impact of humans on an environment, the second and third problems will investigate the impact of disease and climate change on animal populations.

CONCLUSIONS Through focusing in particular on how students might better and more deeply learn knowledge and skills for transfer to new problems and settings as part of theoretically grounded virtual pedagogical experiences, the project seeks to understand how people learn in general and in these specific types of innovative digital media learning environments. The practical significance of the project lies in the fact that there has been little research into how Australian students might learn in these newly

2 http://unity3d.com/

The Dump Site Sample 3

Mosquito


52

available and innovative multi-user virtual environments and the sustainability of MUVEs in the longer term. In addition, there has been little work into opportunities and challenges that in-service Australian teachers might face as they begin to learn about and to integrate technological and pedagogical innovations such as MUVEs into their current and future classes. At the conclusion of the project, there will be sets of pedagogical modules with virtual learning environments for subject topics in science that will have been designed in conjunction with the collaborating teachers and with the Centre of Learning Innovation in the New South Wales Department of Education, which we expect should then be adaptable for more general dissemination to other schools in NSW and in the other Australian states that are linked to the national science curriculum. We also hope the design research approach used in this project might serve as a model for how innovative learning technology and teaching approaches might be iteratively researched in real classroom situations in ways that can, if warranted based on research findings, be implemented more widely in practice longer term.

REFERENCES Bainbridge, W. S. (2007). The Scientific Research Potential of Virtual Worlds. Science, 317(5837), 472-476. Barab, S. A., Thomas, M., Dodge, T., Carteaux, R., & Tuzun, H. (2005). Making Learning Fun: Quest Atlantis, A Game Without

Guns. Educational Technology, Research and Development, 53(1), 86 -107. Barnett, M., Barab, S. A., & Hay, K. E. (2001). The virtual solar system project. Journal of College Science Teaching, 30(5),

300. Becta. (2004). A review of the research on barriers to the uptake of ICT by teachers. Bransford, J., Brown, A., & Cocking, R. (Eds.). (2000). How People Learn: Brain, Mind, Experience and School. Washington

DC: National Academy Press. de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with a computer simulations of conceptual domains.

Review of Educational Research, 68(2), 179 - 201. Henriques, L. (2002). Preparing tomorrow's science teachers to use technology: An example from the field. Contemporary

Issues in Technology and Teacher Education [online serial], 2(1). Jacobson, M. J., June Lee, B. K., Hong Lim, S., & Hua Low, S. (2008). An Intelligent Agent Augmented Multi-User Virtual

Environment for Learning Science Inquiry: Preliminary Research Findings. Paper presented at the 2008 American Educational Association Conference. Retrieved 25/06/2008.

Jacobson, M. J., Kim, B., Miao, C.-H., Shen, Z., & Chavez, M. (2009). Design Perspectives for Learning in Virtual Worlds. In M. J. Jacobson & P. Reimann (Eds.), Designs for Learning Environments of the Future: International Perspectives from the Learning Sciences. New York: Springer.

Johnson, L., Levine, A., Smith, R., Smythe, T., & Stone, S. (2009). Horizon Report : 2009 Australia - New Zealand Edition (No. ). Austin, Tx.

Ketelhut, D., Nelson, B., Clarke, J., & Dede, C. (2010). A multi-user virtual environment for building and assessing higher order inquiry skills in science. British Journal of Educational Technology, 41(1), 56 - 68.

la Velle, L., Wishart, J., McFarlane, A., Brawn, R., & John, P. (2007). Teaching and learning with ICT within the subject culture of secondary school science. Research in Science & Technological Education, 25(3), 339-349.

Lee, D. (1997). Factors influencing the success of computer skills learning among in-service teachers. British Journal of Educational Technology, 28(2), 139-141.

Siorenta, A., & Jimoyiannis, A. (2008). Physics instruction in secondary schools: An investigation of teachers' beliefs towards physics laboratory and ICT. Research in Science & Technological Education, 26(2), 185-202.

Squire, K. D. (2005). Changing the Game: what happens when video games enter the classroom? Innovate: Journal of Online Education, 1(6).

Squire, K. D., Barnett, M., Grant, J. M., & Higginbottom, T. (2004). Electromagentism Supercharged! Learning Physics with digital simulation games. Paper presented at the International Conference of the Learning Sciences, Santa Monica, CA.

Urhahne, D., Schanze, S., Bell, T., Mansfield, A., & Holmes, J. (2010). Role of the Teacher in Computer-supported Collaborative Inquiry Learning. International Journal of Science Education, 32(2), 221 - 243.

Webb, M., & Cox, M. (2004). A review of pedagogy related to information and communications technology. Technology, Pedagogy and Education, 13(3), 235-286.

Webb, M. E. (2005). Affordances of ICT in science learning: implications for an integrated pedagogy. International Journal of Science Education, 27(6), 705-735.

Jarrett, Field, Koppi, Promoting Reflective Dialogue through Group Analysis of Student Feedback

53

PROMOTING REFLECTIVE DIALOGUE THROUGH GROUP ANALYSIS OF STUDENT FEEDBACK Lorna Jarrett, Damien Field, Tony Koppi Presenting author: Lorna Jarrett ([email protected]) Faculty of Agriculture, Food and Natural Resources, The University of Sydney, Sydney NSW 2006, Australia KEYWORDS: scholarship of teaching, survey, qualitative data analysis, teaching principles, discussion

ABSTRACT This paper describes an activity intended to promote scholarship of teaching through small-group discussion of feedback from students. There is a paucity of literature on group reflection of student feedback which this paper aims to address. Reflection on teaching is often a lone activity but this Australian Learning and Teaching Council (ALTC) supported project afforded the opportunity for group reflection by teachers from five institutions during our first project workshop. To provide the data for group analysis, students from the participating institutions completed a survey designed by the project team. A workshop activity was devised in which groups analysed the qualitative survey responses and derived principles for learning and teaching based on their reflection. Evaluation of the activity included workshop participant evaluation forms, feedback from the ALTC project team and evaluator; and the principles developed during the activity. A notable measure of the activity’s impact is that most participants stated that as a result of the workshop, of which this activity was a significant part, they intended to change something about their own teaching. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 53-59, ISBN Number 978-0-9808597-0-6

INTRODUCTION AND RATIONALE FOR THE ACTIVITY This paper describes the process and outcomes of an activity intended to promote scholarship of teaching and learning among soil science academics from five institutions. It involved structured group reflection on student feedback obtained from an online survey and the generation of principles for learning and teaching based on this reflection. The activity is part of a larger project which aims to produce curricular, and cultural change. According to Schön (1987) and Brookfield (1995), cultural change can be brought about through peer discussion and reflection on teaching practices. The student data provided an authentic reason for a disparate, heterogeneous group of teaching staff from five institutions to reflect on and talk about teaching. The format of the activity gave structure and purpose to the conversations. The survey data was current and from the participants’ own students, so it provided a way of bringing the students’ voices into the workshop and getting participants to engage with their students’ points of view. The minimal editing of students’ comments gave immediacy to the activity, which deliberately did not include any conclusions drawn by the researchers in order to encourage active participation by the teachers. The activity also comprised part of the analysis of the qualitative survey data. While methods for analysis of quantitative data are relatively prescribed, qualitative analysis is a subjective and open-ended process (Bogdan & Biklen, 2002). This activity yielded multiple perspectives on the data set, enhancing the credibility of the qualitative survey data analysis.

BACKGROUND This activity is part of a larger project supported by the Australian Learning and Teaching Council (ALTC) involving the Universities of Adelaide, Melbourne, Queensland, Sydney and Western Australia. It aims to develop a national soil science curriculum in response to the needs of students, academic staff, industry and the wider community. The curriculum will be student-centred, encouraging students to take an active role and assume responsibility for their learning. This will involve cultural change for both staff and students. Fullan (1999) states that reflection by individuals, between groups and within organisations, is essential to effecting change in teaching practice. However, Fendler (2003) points out that reflection based only upon the teachers’ own thoughts can fail to confront existing ways of thinking. Loughran (2002) asserts that in order for reflection to be meaningful, a method must be found to enable teachers to see their practice as others do.


54

One of the early stages of curriculum change involves consultation with stakeholders to gather data about the current curriculum (Bath, Smith, Stein & Swann, 2004; Daniels & McLean, 2004; Harden, 2000). Students are key stakeholders in education; therefore consultation with students is vital in helping to ensure that the intended, delivered and received curricula are aligned, alternative viewpoints are heard, unexpected and unintended issues are less likely to be neglected, and misleading results are less likely (Bath et al., 2004; Bruinsma & Jansen, 2007; Plaza, 2007; Wachtler & Troein, 2003). Consultation with stakeholders can take the form of surveys, interviews or focus groups. Our consultation with students took the form of an online survey which generated both qualitative and quantitative data. It was developed by the project team and made available to all students currently enrolled in soil science courses in the participating universities. Our student survey thus served two complimentary purposes: to contribute to the picture of current practice which will form the baseline for curricular change; and to contribute to cultural change by forming the focus of reflective dialogue among teaching staff, thus promoting of scholarship of teaching (Trigwell & Shale, 2004). Andresen (2000) asserts that the activities of intellectual development, inquiry and action must be personal but rigorous. A component of scholarly teaching is the provision of a process through which staff can assess the quality of their teaching. Such a process should be activity-oriented and student-centred; and should lead to a resonance between what teachers aim for and what students experience (Trigwell & Shale, 2004). Trigwell and Shale’s scholarship model assumes a partnership between teacher and student rather than an instructional relationship and thus advocates the inclusion of the student voice as a means to participate in the disciplinary community. Reflecting on survey responses from current students meets these requirements but for this to contribute to scholarship there also needs to be the opportunity for teaching staff to identify how this can be used to effect change. Our activity addressed this need by making students’ voices the focus of staff reflection and group discussion: requiring staff to think critically about the impact of their teaching practices on students and asking them to collaboratively generate learning and teaching principles based on their discussion and reflection.

METHODS AND RESULTS We did not want to adapt existing teaching principles from another discipline as this could be viewed as “preaching” to the participants and would therefore be unlikely to lead to ownership of the project or to result in lasting change. The workshop was attended by about 80% of teaching staff from all participating institutions and comprised several sessions intended to promote engagement and reflection. Sessions were intended to be “hands-on”, with a minimum of passive listening to presentations. Stages in the activity reported here involved the development of the student survey; collation and preparation of the data; implementation of the workshop session; and its evaluation. The preparation and execution of this activity is illustrated in Figure 1 as a series of cycles forming an action learning process (Kemmis & McTaggart, 2001).


55

Figure 1: The workshop activity as an action learning process

DEVELOPMENT OF THE STUDENT SURVEY We obtained Ethics approval for all project activities including the student survey and participation of teaching staff. The authors and the project leader developed the survey which was trialed by members of the project team at the partner institutions. We did not pilot the survey with students because the total number of students available to complete the survey was relatively small (about 450), and a pilot group would not have been available to complete the final version of the survey. We designed the survey with open-response boxes in most questions to allow students to express their ideas fully and avoid constraint by the survey design (Figure 1, cycle 1). The survey was made available online and we received responses from all participating institutions. 107 students responded to the survey and made over 300 comments in total. This represents a response rate of approximately 24 percent. The survey design was informed by Fowler (2002) and the questions are given in Appendix 1.

PREPARATION FOR THE ACTIVITY Prior to the workshop the qualitative data, ie: students’ comments and answers to extended-response questions, were assigned by the lead author to one or more of 22 categories (Figure 1, cycle 1).


56

Some of these categories were determined from the survey questions and some emerged from the data. This qualitative data analysis method was informed by the work of Bogdan and Bicklen (2002) and Boyatzis (1998). For example, the comment: “I prefer assignment or report based assessment, it allows me to explore a subject further and use other resources than learning guides etc.” was categorised under “assessment”, “meaningful learning” and active learners” The purpose of this was to sort the raw data into sets, each relating to a particular category, which could form the focus for discussion of that category. The next stage in the process involved choosing which of the 22 categories would be the focus of group discussions: our workshop activity had four groups, so we required four categories. Our selection criteria were: interest to participants; potential for contribution to the development of teaching principles; and alignment between the outcomes of the activity and the desired outcomes of the ALTC-supported project. The categories we chose were: “active learners”, “careers”, “most effective activity” and “effective learning” (Figure 1, cycle 2). Comments which had been placed into these categories were extracted to form a list for each category: each list contained between 18 and 25 comments. Any potentially identifying information was removed and spelling corrected but no other changes were made. The four sets of student comments were printed and the pages cut into strips, with each strip containing a single comment. This was to prevent the participants perceiving them as an ordered list with the associated primacy/recency (Jersild, 1929) effects; to help participants to focus on each comment individually; and also to allow participants to physically organise the comments into groups to aid their analysis.

THE ACTIVITY Workshop participants were assigned to four groups, with at least one member of each institution in each group wherever possible (Figure 1, cycle 2). Each group was issued with a set of student comments, a set of instructions comprising two tasks and a list of possible questions to stimulate discussion. The instructions for the four groups were broadly similar, but the focus questions and tasks were tailored to the category being reflected upon. As an example, the instructions for the category “active learners” are shown below:

TASKS - ACTIVE LEARNERS 1. Write down words and phrases to summarise what the data suggests about fostering active learners. 2. Can you formulate two teaching principles to support and encourage students to take responsibility for their learning?

POSSIBLE QUESTIONS TO FOCUS DISCUSSION How would you define an active learner? Are our students active and self-motivated learners? What factors are preventing them from taking a more active role in their learning? What things are we doing to help them take responsibility for their learning? Are there any contradictions in what the students say? Can you explain these? What skills do students need to engage in active or self-directed learning? What topics, activities and year groups is active or self-directed learning suitable / unsuitable for? Although some groups felt that there was not enough time, all groups were able to develop teaching principles. Each group summarised their discussion on butchers’ paper, generated two teaching principles for their category and presented their findings to the other participants.

EVALUATION The activity was evaluated in terms of: Principles derived from the student comments by each group The intention, expressed by most participants, to make changes to their teaching as a result of the

workshop Feedback from the participants, ALTC project team members and project evaluator. The most tangible outcomes of the activity were the principles drawn up by each group during the activity (Figure 1, cycle 3). Less tangible outcomes include focused reflection and discussion between teaching staff, engagement with and critical evaluation of students’ feedback and a feeling of ownership of the project. Evidence of these was gathered through observation of participants during


57

the activity, participants’ workshop evaluations and comments made by the ALTC project team and evaluator. Of the sixteen participants who completed workshop evaluations, thirteen agreed or strongly agreed that they would change something about their teaching as a result of the workshop. Given that this workshop was intended simply to begin the process of cultural change in teaching, this is a powerful result. Participants made the following comments on their evaluation forms: “Interpretation of student comments – good because we were interpreting / analysing actual feedback” “Student feedback information and good teaching principles were very useful”. “It was excellent to share ideas about teaching with other soil scientists” “Props were good, some thought had gone into “helping” us contribute at sessions” “Students’ feedback – too much to do in the given time” These comments demonstrate the value of both the activity’s structure and use of recent student feedback but underline the importance of providing enough time for participants to engage fully with it. The ALTC project team discussed the activity the next day as part of their workshop appraisal. This discussion was audio-recorded and the following remarks were made about the activity: “The group activities worked surprisingly well. What we did was effective and generated a depth of discussion that perhaps we wouldn’t have achieved otherwise”. “I thought it went extremely well and I’ve been to many of these sessions. People were obviously genuinely engaged – it wasn’t just a talk-fest”. “The preparation really made the day work. The structure made it so that people wanted to come. The structure was good – that was an important part of why it worked”. These quotes confirm the depth of engagement and reflection that occurred and underline the importance of structure and purpose in the design of activities in helping to achieve this.

CONCLUSIONS AND RECOMMENDATIONS Reflection by individual teachers on feedback from their students is a common practice (Hoban, 2000; Hoban & Hastings, 2006; Wickramasinghe & Timpson, 2006) but we believe that the cross-institutional group reflective activity presented here has the additional advantages of greater anonymity for students, feedback from a wider student group and staff collaboration. The on-line survey was clearly identified cross-institutional. This may have encouraged students to be more forthright in their comments as opposed to completing an evaluation in their ‘classroom’ because of the much larger number of students participating. Rather then reflecting on data from a group of students they had recently taught, the participants reflected on feedback from all participating institutions, representing (within sampling limitations) the perceptions of the whole student body. This may have had the effect of de-personalising the experience and making it less threatening. The collaboration of teachers from different institutions, each with their own experiences, culture and concerns, may have increased the opportunities for questioning assumptions and breaking out of entrenched ways of thinking (Hoban & Hastings, 2006). Evaluating our action learning cycle, we are satisfied that each cycle produced information of sufficient quality to inform the subsequent cycles. Although not indicated in Figure 1, cycle 3 resulted in a list of 10 teaching principles that have been distributed to the participants in the project for comment. We emphasise that allocation of time is critical, as is the amount of stimulus material as some of the groups did not complete their discussion in the time available. We also recommend written and/or audio recording of the discussions which may be shared with other participants enabling them to observe the conversations held by other groups and the detail behind the principles they developed. Audio recordings would also reduce the burden on participants of simultaneously engaging in discussion and making notes. Finally, to develop the outcomes from cycle 3 there is potential to develop a strategy for follow-up: for example inviting participants to record changes that they wanted to make as a result of the workshop and soliciting feedback on whether they had made the intended changes. Trigwell and Shale (2004) caution that scholarship does not end with discussion but depends critically on putting ideas into action.


58

FUTURE RESEARCH The teaching principles developed during our workshop will be used in the next action cycles of our project. The workshop activity framework will be used to elicit feedback from employers in our next workshop and we will endeavour to determine whether the activity has resonates with them.

ACKNOWLEDGEMENTS We would like to acknowledge the Australian Learning and Teaching Council for funding the project of which this activity is a part, and also the invaluable contribution of the workshop participants.

REFERENCES Andresen, L. W. (2000). A useable, trans-disciplinary conception of scholarship. Higher Education Research & Development,

19(2), 137–153. Bath, Smith, Stein, & Swann. (2004). Beyond mapping and embedding graduate attributes : bringing together quality assurance

and action learning to create a validated and living curriculum. Higher Education Research & Development, 23(3), 313-328. Bogdan, R. C., & Biklen, S. K. (2002). Qualitative Research for Education: An Introduction to Theories and Methods (4th ed.).

Allyn & Bacon. Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development. Sage Pubns. Brookfield, S. D. (1995). Becoming a Critically Reflective Teacher. Jossey-Bass Higher and Adult Education Series. Jossey-

Bass, Inc., 350 Sansome St., San Francisco, CA 94104 Bruinsma, M., & Jansen. (2007). curriculum mapping: integrating multiple perspectives on the curriculum. curriculum and

teaching, 22(1), 25-45. Daniels, & McLean. (2004). Integrating Technology into Teacher Education Through Curriculum Mapping: An Update on Year

Two. In Proceedings of Society for Information Technology and Teacher Education International Conference 2004 (pp. 2089-2094). Presented at the Society for Information Technology and Teacher Education International Conference (SITE) 2004, Atlanta, GA, USA.

Fendler, L. (2003). Teacher Reflection in a Hall of Mirrors: Historical Influences and Political Reverberations. Educational Researcher, 32(3), 16-25. doi:10.3102/0013189X032003016

Fowler, F. J. (2002). Survey research methods. Sage. Fullan, M. (1999). Change forces: the sequel. Routledge. Harden, R. (2000). Curriculum mapping: a tool for transparent and

authentic teaching and learning. Medical researcher, 23(2), 123-137. Hoban, G. (2000). Making practice problematic: Listening to student interviews as a catalyst for teacher reflection. Asia-Pacific

Journal of Teacher Education, 28(2), 133–147. Hoban, G., & Hastings, G. (2006). Developing different forms of student feedback to promote teacher reflection: A 10-year

collaboration. Teaching and Teacher Education, 22(8), 1006-1019. doi:10.1016/j.tate.2006.04.006 Jersild, A. (1929). Primacy, recency, frequency, and vividness. Journal of Experimental Psychology, 12(1), 58-70.

doi:10.1037/h0072414 Kemmis, S., & McTaggart, R. (2001). Participatory action research. In N. Denzin & Y. Lincoln (Eds.), Handbook of Qualitative

Research (Second ed., pp. 567-605). Thousand Oaks: Sage. Loughran, J. J. (2002). Effective reflective practice: In search of meaning in learning about teaching. Journal of Teacher

Education, 53(1), 33. Plaza. (2007). Curriculum Mapping in Program Assessment and Evaluation. American Journal of Pharmaceutical Education,

71(2). Schön, D. A. (1987). Educating the reflective practitioner. Jossey-Bass San Francisco. Trigwell, K., & Shale, S. (2004). Student learning and the scholarship of university teaching. Studies in Higher Education, 29(4),

523. doi:10.1080/0307507042000236407 Wachtler, C., & Troein, M. (2003). A hidden curriculum: mapping cultural competency in a medical programme. Medical

Education, 37(10), 861-868. doi:10.1046/j.1365-2923.2003.01624.x Wickramasinghe, S., & Timpson, W. (2006). Mid-Semester Student Feedback Enhances Student Learning. Education for

Chemical Engineers, 1(1), 126-133. doi:10.1205/ece06012


59

Appendix 1: Survey questions No. Question text Qualitative or

quantitative? Answer format Comment

box? 4 Activities and materials involved real-life or

realistic scenarios and/or case-studies. Both Likert scale: strongly

disagree strongly agree

Yes

5 Subject activities and materials involved input from industry

Both Likert scale: strongly disagree strongly agree

Yes

6 The subject activities involved me applying my knowledge to give explanations, justify decisions, make predictions or suggest solutions to problems.


Yes

7 I was able to contribute to the learning agenda e.g. by choosing experiments or essay topics or giving feedback during lectures so the lecturer could focus on what I needed to learn.


Yes

8 The assessments allowed me to demonstrate the knowledge and skills I had learned.


Yes

9 I think the subject helped prepare me for future employment in a soil science related area


Yes

10 How much of the subject content involved rote learning (memorising facts and figures)?

Both Likert scale: Percentages in 20% steps

Yes

11 Apart from laboratory work and fieldwork, how much of your subject activities involved group work or group discussions?


Yes

12 How much of the time did you spend in a passive role such as listening to lectures, following set procedures in laboratories, solving set problems in tutorials?


Yes

13 The way the subject is taught suits the way I like to learn


Yes

14 If there are differences between how the subject is delivered and what you want, please tell us more.

Qualitative Extended response N/A

15 I learn best from:


Yes

16 It's easy to discuss my work with the teaching staff


Yes

17,18 Please tell us about the learning activity that was most effective for you and why it was so effective


19 Is there anything else you would like to tell us about your experience with soil science?



60

DOES A CONFERENCE ACT AS A CATALYST FOR FURTHER PUBLICATIONS AND COLLABORATIONS? A PILOT STUDY OF A SMALL SCIENCE AND MATHEMATICS EDUCATION CONFERENCE. Hazel Jonesa, Alexandra Hugmanb Presenting Author Hazel Jones ([email protected]) aUniServe Science, The University of Sydney, Sydney NSW 2006 Australia bSUPER, School of Physics, The University of Sydney, Sydney NSW 2006 Australia KEYWORDS: science and mathematics education, networking, conference, collaboration.

ABSTRACT Many university academics spend time and money attending national and international conferences each year. This initial longitudinal study seeks to investigate the perceived and actual benefits and outcomes specifically from delegates attending the 2009 UniServe Science Conference. This is a relatively small three day annual conference for science and mathematics educators. Data was collected from an online survey immediately after the event and another survey six months later. This revealed information concerning the demographics of the delegates and sought detail on the synergies and collaborations, articles and grants emerging in the short term from such a conference. Networking emerged as the main benefit along with the opportunity to find out about new ideas to implement in their own teaching and research. Delegates indicated they had used ideas from the conference and shared these with others. However there was a decline in actual submissions to journals compared with the expectations of delegates immediately after the conference. Findings lead to alternative formats for conferences and suggest changes to allow maximum benefits for all stakeholders. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 60-65, ISBN Number 978-0-9808597-0-6

INTRODUCTION Institutions and individual staff members invest considerable resources in conference attendance every year. For example, this three day conference, including airfares, accommodation, meals and registration fees represents an investment of at least $800 per delegate in addition to the hours spent in preparation of papers and attendance at the event. The purpose of this study was to determine what delegates considered to be the short term and longer term benefits for themselves, colleagues and their institution as determined by change to practice, implementation of new ideas in research and/or teaching, publications and networking and collaboration opportunities. The results of the survey will influence the format of future conferences with the overall aim of improving outcomes for delegates. We were also interested to learn what were the unique aspects of this conference that attracted delegates and this aspect will be discussed in future papers, once the final survey is completed. The activity of conferencing is defined by Nadler and Nadler (1987) as "a group of people who come together for a variety of purposes and vary in size and duration".(p.2) Generally delegates attend a conference because they have identified a need to broaden their knowledge by hearing from experts in a particular field, wish to find out recent developments/research, network with people in similar situations and with similar interests, or to share problems and accomplishments (Muir-Cochrane, Lawrence & Zeitz, 1991). In the university sector this has been a well recognised and intrinsically valuable activity for many years. A literature search identified guidelines and recommendations concerned primarily with the running and organisation of the conference process over the past 30 years (Collins, 1985; McDonald, Neat,

Jones, Hugman, Does a Conference act as a Catalyst for Further Publications and Collaborations?

61

Tanyu, Mason, Taylor-Ritzler, Blanton, & Reeves, 2004; Muir-Cochrane et al., 1991; Nadler & Nadler, 1987; Seadle, 2009); many of which advocated the importance of evaluation. Conference evaluation samples were available on the web, but these also focused on the logistics and aesthetics of the venue. Mueller (1982) questioned whether the evaluation sheets merely gave a ‘happiness report’ rather than rating the effectiveness of the event, and advocated the distribution of a further follow-up survey six months to a year after the conference to assist with this assessment process. Further reading identified some research exploring virtual conferencing (Bell & Shank, 2006; Farkas, 2006) as an economic option with respect to both time and cost. In contrast Listservs (electronic mailing lists) blogs and wikis, with a focus on a particular topic and regularly updated, are continuous and therefore current and can provide immediate feedback. Hardesty and Sugarman (2007) carried out a ‘Academic Librarians, Professional Literature and New Technologies’ survey and 95% of respondents identified Listserv as their main method of keeping abreast with literature in their discipline. Little was reported on the importance of the outcomes of a conference and how it had benefited the practice of the delegates. The exceptions were the field of academic librarians, who reported generally on the benefits of networking, discipline related information and research and social interactions (MacDonald et al.,2005; Tomaszewski & MacDonald, 2009); and the medical field (Davis, O’Brien, Freemantle, Wolf, Mazmanian & Taylor-Vaisey,1999). The latter reported on 16 events involving either didactic or interactive sessions, or a mix of the two styles of pedagogy, and investigated whether any change in the delegates’ practice was noted. No change in practice was recorded from the didactic sessions, however both the interactive and the mixed sessions showed statistically significant positive outcomes in practice.

THE STUDY The annual UniServe Science Conference focuses on innovations in science and mathematics education research. Its aim is to provide an opportunity for tertiary science and mathematics educators to come together, share ideas and keep up to date on emerging research This longitudinal pilot study sought to investigate the effectiveness of the 2009 conference with a view to improving the experience of the 100 plus delegates but more specifically to improve the benefits to teaching and research, hence developing a more productive community. These extrinsic benefits may take the form of collaborations, adopting technologies or pedagogies, or publications. Qualitative and quantitative data was collected from a series of online surveys. In previous years a short evaluation survey was conducted at the conclusion of each conference, which focussed only on logistics and the “feel-good” aspects of the conference. 2009 was the first time a more in-depth or longitudinal evaluation was undertaken. In 2009 the three day conference included keynote lectures, workshops, seminars and social activities. This format has previously been identified as providing the optimum mix for change in practice (Davis et al., 1999). Delegates were invited to complete an online survey at the conclusion of the conference, and were informed that there would be two further follow-up surveys at six (Phase 2) and 12 (Phase 3) month intervals to assess the further effectiveness of the event as recommended by Nadler and Nadler (1987) but not reported on elsewhere. The online, anonymous survey was developed using ‘Survey Methods’ software, and access was sent to all delegates using email. It was estimated to take 15 minutes to complete, and included multi-choice and opportunity for extended responses. This paper discusses only the Phase 1 and 2 surveys, as the Phase 3 survey is yet to be distributed.

RESULTS

PHASE 1 The first of the voluntary, anonymous online surveys was sent out by email to each of the 115 delegates during the week following the conference. Questions sought information concerning the logistics and process of the event, but also asked the delegates to predict how their future activities might be influenced and responsive to the experience. There were 75 responses, 65% of the original cohort, whose demographic is shown in Table 1. Almost half of these delegates had not attended previous UniServe Science conferences, and 70% of


62

the respondents had been presenters. Over 80% claimed to have attended because it was relevant to their area of research and it had been good in the past. Table 1: Demographic details of survey responses from the UniServe Science Conference 2009 (Total delegates = 115 Phase 1 N=75, Phase 2 N=37)

Concerning the immediate outcomes from the conference, delegates were asked to comment on their likelihood of sharing ideas with colleagues or publishing from their presentation. Question: ‘What ideas will you take back to your workplace and share with your colleagues?’ From the 75 survey respondents 40 extended answers were received to this question. They gave rise to a range of topics and coding the responses broadly identified that 75% of ideas were related directly to teaching and learning practice whilst 10% highlighted research ideas. Ideas linked with the conference theme of motivation and engagement were mentioned specifically in over 30% of the responses concerning students and once concerning staff colleagues. The keynote lecture on the educational theories on motivation was specifically applauded in 10% of responses. In a further 10% of comments innovative technologies were included as useful tools to be adopted. Generally, two respondents expressed pleasure in the interest shown and advice by delegates, and other comments highlighted the range of ideas and the collaborative nature of the conference. Question: Are you considering revising your work and publishing it in an educational research

journal? Responses indicating anticipated outcomes from the conference showed that 39 delegates (n=75) were considering the revision of their presentation with a view to publication within an educational research journal. Of these twelve respondents were able to name the journal likely to be targeted for their work, generally choosing ones that were ranked highly, or through recommendation. There were only two mentions of the UniServe Science publication CAL-laborate International and one comment highlighted the lack of journals publishing work on ‘chemical education and teaching practice’. Three noted that their research was ongoing and so not suitable for publication yet. The value of social networking was highlighted in many responses, and more opportunities for formal networking was requested as an improvement to be adopted at future conferences For example “a networking session, where there is time allocated for people to legally sneak away in little groups for coffee and make plans, without missing any talks.” “Wanting to keep talking to the people who were there - but I guess we do have to return to work!”

PHASE 2 The second survey was emailed to delegates six months after the conference. The questions mirrored those from Survey 1, although questions relating to the logistics and satisfaction levels were omitted. Questions were tailored to prompt the delegates to reflect on how their recent activities had been influenced by their experience at the 2009 UniServe Science Conference. There were 37 responses, which is 50% of the original respondents (32% of the original conference cohort) whose demographic is shown in the comparison column of Table 1. Could this reduced response rate be an indication that many delegates do not think about long term benefits from attending a conference and rather just see attending a conference as being a stand-alone benefit of their position?

Delegate Total Delegates

Phase 1 Phase 2 (6 months later)

Delegate Total Delegates

Phase 1 Phase 2 (6 months later)

Male 48 27 10 Academic 98 54 31 Female 67 44 28 General 7 7 3 No reply 4 Student 10 10 4 Age <30 9 3 30-40 14 7 First

attendance 32 13

41-50 20 12 Presenter 73 54 26 >50 28 16 No reply 4


63

Two questions asked delegates to comment on their continued contact with other delegates from the conference. The data is summarised in Table 2. About 70% of respondents stated that they had maintained links, many with already established relationships and half of the comments acknowledged the making of new contacts. 40% of respondents acknowledged that collaborations had taken place, several listing existing groups. These figures indicate that this is a positive aspect and that delegates have generally been able to develop and cultivate existing and new relationships. One response indicated that they had “established a number of contacts to begin collaborative work with Sci Ed”. Conversely, there was one first-time delegate who commented that “as a first-time attendee being my first time, I found it quite difficult to actually meet and have meaningful conversation.” This is one area that can be looked at for improvement at future conferences. Table 2: Summary of the number of respondents indicating outcomes achieved following the UniServe Science conference 2009 (Phase 1 N= 75, Phase 2, N=37)

Descriptor Phase 1 Phase 2 I have kept in touch with other delegates from the 2009 conference 25 I have collaborated with other delegates from the 2009 conference 15 I have used ideas/knowledge gained in my teaching 20 I have used ideas/knowledge gained in my research 15 I will/have shared ideas/knowledge gained with my colleagues 40 27 I will/have submitted a revised paper of my presentation for publication 39 4

Three further questions requested delegates to comment on their use of any knowledge or ideas, in the preceding six months. Question: Have you used any ideas/knowledge gained from the 2009 UniServe Science

Conference in your teaching? Over 50% of the respondents indicated positively, nine specific examples were cited, two of which included technology tools. Areas particularly mentioned were those of motivating students, improving laboratory sessions with ASELL, and assessment techniques. Two comments identified that insights provided by attendees at their presentation had influenced their practice. One comment was echoed several times in the negative responses “but this is due to lack of time, not lack of worthwhile material”. Question: Have you used any ideas/knowledge gained from the 2009 UniServe Science

Conference in your research? This question received a positive response from 40% of the cohort. Extended answers revealed a variety of outcomes from the utilisation of theories and ideas to extend personal research, stimulation of new areas of research and grant applications. One comment in this section opined that “UniServe Science is a bit more ‘show and tell’ than theoretical, so it’s not as useful for research as it is for meeting people”. However, to illustrate the diversity of responses, another commented ‘Our development and dissemination of teaching strategies under an ALTC grant has hinged critically on input and involvement gained via the UniServe Science conference. Subsequent conferences and teaching fora have gained us only a fraction of the information, interest and engagement offered by the UniServe Science conference’. Question: Have you shared any ideas/knowledge gained from the 2009 UniServe Science

Conference with your colleagues? This question received the largest positive response, from 75% of the replies. From the 18 extended comments five related to introducing new technology and one referred to yet another ALTC grant application. Several delegates related opportunities to present their work and further feedback from the conference to their work colleagues. A further 75% of the responses agreed to recommend the 2010 conference to their colleagues. One comment stated ‘In a discussion of good education conferences UniServe [Science] was mentioned as being a good conference to attend’ and another ‘Highly recommended …very valuable for both specific content/presentations and as a networking opportunity…’ One further question involved the extent to which publications had been identified as an important outcome of the conference as it was linked to university funding. Question: Have you submitted a revised paper of your presentation for publication? Of the 26 responses to this question four delegates had already submitted papers to journals within the six months following the conference. One of these had been accepted with minor revisions. The


64

remaining three authors had been invited to submit revised papers to the UniServe Science publication International Journal of Innovation in Science and Mathematics Education (IJISME) and these were under review. In Phase 1 there had been 39 delegates hoping to submit a revised paper for publication (Table 2).

DISCUSSIONS AND CONCLUSIONS The extrinsic outcomes of conferences, in line with the literature, are social networks (Muir-Cochrane, Lawrence & Zeitz, 1991) and change in practice (Davis et al., 1999). Due to recent university funding demands, this study also sought to identify journal publications as being a third important outcome. The main benefits of the 2009 UniServe Science Conference, as described by delegates are the informal and formal networking, synergies and collaborations. These are a key activity of the conference itself, but also clearly extend for the period up to six months. Three grant applications were reported as having been enhanced or initiated by collaborations ensuing directly from the conference. This is an aspect that could be further exploited through providing greater opportunities for formal and informal discussions along these lines during the conference and facilitating ongoing discussion groups. Teaching has been identified as a key area for changes in practice, and, unlike Davis et al. (1999), the keynote lecture was explicitly mentioned as influencing change. Technologies also featured as a popularly adopted strategy for increasing student engagement. These areas of interest could be further explored during the Ideas Exchange section of the conference and/or workshops providing opportunities for delegates to try innovative technologies and practices. In this regards it will need to be remembered that delegates will have different perceptions of what is innovative, depending on their previous experiences with technologies. These differences are echoed in the following opposing comments from delegates in the Phase 1 survey “This year, I had the impression that there weren't that many "new" ideas. That may be just because - now, after a few years of attendance - I've heard most of them before.” and “Sharing ideas, relevant, innovative and interesting practices” The number of delegates who have shared ideas and knowledge gained at the conference with their colleagues is also a positive outcome. Whilst the actual number of delegates who indicated they have shared decreased between Phase 1 and Phase 2 the percent of positive responses, in relation to total number of responses has actually increased. This is most likely a reflection that delegates have had opportunities to hold these discussions with colleagues. It is also an indication that the benefits of the conference reach to a much wider audience that those who actually attend the conference. One way of assisting delegates to share knowledge and information could be to produce a summary document outlining the key points discussed during the conference. The logistics of producing this, including determining who contributes to the development of the document will require further thought. The most disappointing outcome was the conversion rate of conference papers to journal articles with only four submitted articles identified in the Phase 2 survey, compared with the expected 39 indicated in the Phase 1 survey. This may reflect the relatively short time frame of six months, and supports the implementation of Phase 3 of the study. It is also interesting to note that the successful papers were invited submissions. This suggests that a continuation of a Special conference issue of the journal would be a worthwhile exercise as an invitation to submit appears to provide the necessary motivation to develop a paper. The final survey, Phase 3, may see some of this balance redressed as work commitments change through the semesters and delegates will have had further time to develop their research and submissions. It would appear that, for the UniServe Science Conference, there are many benefits of the traditional format mainly due to the social networking involved. All delegates participating in the survey agreed that they would consider attending the 2010 event, which implies that, regardless of constraints of time and expense, the overall experience is seen as valuable. There are some constraints on the inclusion of any of the suggested changes to the format of the conference.


65

The final phase of the evaluation will be completed at the end of September, leaving insufficient time to implement any changes for 2010

Since the 2009 conference, there has been a change in the team and management structure of UniServe Science which many have some effect on the introduction of these suggested changes

It is though hoped that at least some of these will be able to be implemented in 2011. Phase 3 of the study will provide further information concerning the longer term outcomes of such an event for science and mathematics educators. Evaluation of this final phase will hopefully provide further insights which will lead to enhancements of the format for future conferences. Possible areas for further studies include comparisons with larger conferences and their outcomes and comparison of this survey with surveys of future conferences, with enhanced format, to determine whether the changes suggested from the current research do prove to be of long term benefit to delegates.

ACKNOWLEDGEMENTS Ethics approval was sought and granted from The University of Sydney Human Ethics Committee for this evaluation. (No: 12105)

REFERENCES Alaimo, R. (2004). Top Six Reasons to Attend a Conference, Knowledge Quest 33 (1) 34–35. Bell, S. & Shank, J. D. (2006) Conferencing @ Your Computer: The Ins and Outs of Virtual Conferences, Library Journal 131

(4) 50–52 Collins,M.(1985). Quality learning through residential conferences, New Directions in Continuing Education. 8. Davis, D., O’Brien, M. A. T., Fremantle, N., Wolf, F. M., Mazmanian, P. & Taylor-Vaisey, A. (1999). Impact of formal continuing

medical education: Do conferences, workshops, rounds and other continuing education activities change physician behaviour or health care outcomes? Journal of the American Medical Association 282 9 pp. 867-874. Retrieved from www.jama.com on 22 June 2010.

Farkas, M. G. (2006). A Glimpse at the Future of Online Conferences, American Libraries 37 6 p. 28. Hardesty, S. & Sugarman, T. (2007). Academic Librarians, Professional Literature, and New Technologies: A Survey, The

Journal of Academic Librarianship 33 (2) 196–205. Matsuo, Y., Tomobe, H., Hasida, K. & Ishizuka, M. (2003). Mining social network of conference participants from the web.

Proceedings IEEE/WIC International Conference on Web Intelligence. McDonald, K., Neat, J., Tanyu, M., Mason, G., Taylor-Ritzler, T., Blanton, S. & Reeves, E. (2005). Evaluation 2004:

International Attendees. Chicago, Il.: American Evaluation Association. Muir-Cochrane, E., Lawrence, K. & Zeitz, K. (1999). Striking a balance: Facilitation of process at the 30th national AAACE

conference. Australian Journal of Adult and Community Education, 31, (2) 119-129. Mueller, G. (1982). Successful conference programming methods. Your guide to effective planning and administration. US:

Fern Publications. Nadler, L. & Nadler, Z. (1987). The comprehensive guide to successful conferences and meetings. San Francisco: Jossey-

Bass. Seadle, M. (2009). Conference contrasts. Library Hi Tech 27 (1) 5-10. Tomaszewski, R. & MacDonald, K. I. (2009). Identifying subject specific conferences as professional development opportunities

for the academic librarian. Journal of Academic Librarianship, 35 (6) 583-590


66

FORMING GROUPS TO FOSTER COLLABORATIVE LEARNING IN LARGE ENROLMENT COURSES

Gwendolyn A. Lawriea, Kelly E. Matthewsb, Lawrence R. Gahana

Presenting author Kelly E Matthews ([email protected]) aSchool of Chemistry & Molecular Biosciences, The University of Queensland, Brisbane Qld 4072, Australia

bTertiary Education Development Institute, The University of Queensland, Brisbane, Qld 4072, Australia

KEYWORDS: Collaborative learning, large classes, group formation, group process, active learning

ABSTRACT Assessed group tasks are becoming more prevalent in large undergraduate courses as a means of creating active, collaborative learning environments that foster student engagement and build team-work and communication skills. However, introducing group work presents challenges around task design, implementation, management and marking that differ significantly from individual-based assessment tasks. This paper focuses specifically on the role of team formation in collaborative learning tasks, which is situated in a broader, on-going study of interdisciplinary scenario-inquiry tasks in large enrolment science courses. A mixed-method design, based on grounded literature, examined student perceptions of assessed group tasks from two student cohorts completing a task under similar conditions with separate group formation criteria. Initial findings indicate that deliberately formed students groups are preferable to randomly formed groups, influencing student perceptions of group work and their subsequent learning outcomes. Results are interpreted within the context of current literature on group formation and collaborative learning. Implications for forming groups within collaborative learning tasks are presented, along with recommendations for further research. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 66-71, ISBN Number 978-0-9808597-0-6

INTRODUCTION Large (>1000) 1st year classes pose a challenge to instructors who aim to enhance learning in cohorts where diversity in learners’ abilities, interests and backgrounds is a common occurrence. In order to overcome this diversity many instructors have introduced collaborative learning tasks. The introduction of such tasks is based on literature which recommends reform in course design using high impact learning practices to enhance engagement (Kuh, 2003) and the promotion of active learning (Prince & Felder, 2006). Collaborative learning environments offer the opportunity for students to develop shared understanding of concepts (Kagan, 1992; Johnson, Johnson & Smith, 1998; Smith, Sheppard, Johnson & Johnson, 2005; van den Bossche, Gijselaers, Segers & Kirschner, 2006). Further, there is strong evidence that effective collaboration promotes mutual knowledge construction through shared discourse resulting in increased performance (van den Bossche et al., 2006). There is also acceptance that these learning environments are social constructs where the group function depends on interpersonal relationships and individual values (Gillespie, Rosamond & Thomas, 2006; van den Bossche et al., 2006). The effectiveness of a group is not guaranteed by simply putting people together and, with this recognition, successful group practices have been recommended (Felder & Brent, 2001; Smith et al., 2005). Insights into the factors which promote positive interdependency in groups as they reach solutions or formulate new ideas are still emerging as are tools to evaluate these processes (Summers, Beretvas, Svinicki & Gorin, 2005). Indeed there is strong evidence that group formation, and the role of the instructor, are critical in the success of collaborative learning (Gillespie et al., 2006). Students’ perceptions of the learning in collaborative groups relate to the attitudes and values of their team members in terms of academic aspirations, respective contributions and task outcomes. However, most literature studies report quantitative data gained from studies that explore group processes, perceptions and engagement in small to moderate class sizes (< 200) (Phipps, Phipps, Kask & Higgins, 2001; Smith et al., 2005). The paucity of qualitative studies offers the potential to provide new insights into collaborative learning processes (Coll & Chapman, 2000). In this study we report the evaluation of a collaborative learning activity in large general chemistry classes through a mixed methods approach. This paper represents part of a larger, on-going study

Lawrie, Matthews, Gahan, Forming Groups to Foster Collaborative Learning

67

into the effectiveness of interdisciplinary scenario-inquiry tasks (IS-ITs) as a means of enhancing students’ engagement and appreciation for the interdisciplinary nature of science whilst also catering for the overwhelming diversity of learners’ academic abilities, interests and motivations in first year. An integral element of this study is fostering collaborative learning as students work in teams of four over an eight-week period to complete the tasks. This paper focuses on the process of developing those student teams and is based on the following research question: How will student perceptions toward collaborative learning and their own learning gains be influenced by group formation within an on-going assessment task (IS-ITs)?

METHODOLOGY The context was a first year Chemistry course containing 1100 students enrolled in up to 40 separate programs. The major programs represented promote a professional career identity, viz science, medicine, pharmacy, biomedical science, engineering, biotechnology. A collaborative learning activity was designed to involve students in collaborative groups where they could engage in discourse and identify the chemistry concepts underpinning their assigned topic. This task was initiated in 2008 with adaptations implemented in 2009 as a result of student feedback. In the first iteration, focus groups and surveys were completed to evaluate aspects of the course reform. As a result of the evaluation of the initial intervention, issues were identified relating to the way the students engaged in the task and progressive modifications were made to the task to address these issues. Groups of four students, and topics, were assigned randomly in 2008. In 2009, more attention was paid to group formation based on grounded theory (Johnson et al., 1998; Kagan, 1992). Thus, in 2009 students were clustered by program so that they were working together both with colleagues with common career aspirations and with chemistry topics relevant to these programs. Heterogeneous groups of four were assembled based on mixed academic ability (Felder & Brent, 2001; Kriflik & Mullan, 2007), with gender dispersed to minimise the number of same gender groups, and distribution of international students to address simultaneous hurdles related to English being a second language and to improve their integration into a new environment (Kavanagh & Crosthwaite, 2007). Significant scaffolding was implemented to shift student perceptions of the assessment in relation to learning outcomes (shifting from conceptual gains to teamwork and creativity). Resources were also provided to help students work effectively in groups and address interpersonal issues if they arose. The University’s ethics committee for research involving human subjects approved ethical clearance for this study. Further, the study was funded by the Australian Learning and Teaching Council. A mixed methods approach was adopted with all data collected via an online survey with Likert scale and open response questions. Given the nature of the research question, surveys were considered to be effective instruments for collecting attitude data from large numbers of students. In this study, student perception is used as an indirect measure of student learning, a common practice in higher education research (Kuh, 2003; Seymour, Wiese, Hunter & Daffinrud, 2000). The survey was based on the work of Seymour et al. (2000) and the design routinely used across STEM disciplines in the US and a recommended approach of the National Science Foundation (SALG, 2010). Students were invited to complete the surveys by email, participation was voluntarily and all responses were de-identified. Reminders were sent to students to encourage participation, however no incentives were offered. Threshold requirements for acceptable survey response rates were achieved across all surveys. The responses were monitored by the central university unit administering the surveys in both years of data collection. The quantitative and qualitative data were analysed using standard research software (SPSS and QSR NVivo). Descriptive statistics, including mean and standard deviation (SD), are used to describe student perceptions. The magnitude of change from 2008 to 2009 was assessed using Cohen's d effect size analysis, which is independent of sample size and is considered a more robust indicator in educational research than significance testing (Thompson, 1998). A value of 0.20 is considered a small effect, 0.50 is considered a medium effect, and 0.80 is a large effect (Cohen, 1992). Recurring themes in qualitative data were identified by two analysts independently and cross-referenced to inductively code emerging ideas. The respective counts were cross-correlated to student responses to quantitative questions in NVivo.


68

RESULTS Likert scale responses to the question, How much did the group learning/assessment task help your learning?, were cross correlated with open-ended responses to the question, Please explain how the group learning/assessment task helped your learning? Several themes emerged but the greatest number of references was to the theme of ‘group function’. The results from 2008 and 2009 are presented in Figure 1. The qualitative data relating to group function used to construct Figure 1 were explored further by subdivision into inductively coded daughter nodes which identified factors such as: peer academic aspiration, familiarity with team members, individual contribution, logistics of interactions, and impact of peer assessment as important.

Figure 1: Cross-correlation of Likert Scale responses with open-ended responses to question, 'How much did the group assessment task help your learning?' using a five point, ascending Likert scale on x-axis. As part of a diagnostic survey to investigate the preparation of students for the 1st year chemistry courses and to identify transitional factors, students were asked, at the beginning of their 1st year, whether they preferred to work alone or work with others on assessment tasks. Post-task data in response to this question were collected as part of the course evaluation (Table 1). In both years, pre/post survey results revealed that student preferences in working alone on assessment tasks became more positive during the semester. However, the effect size is small in both years (Cohen’s effect size analysis) indicating that student attitudes to working in group assessment tasks did not alter substantially between 2008 and 2009. The strategy of moving group formation from random assignment to a criteria-based process made little difference to student attitudes. Table 1: 2008 and 2009 data from diagnostic (delivered in week 1-2 of semester 1) with end of course evaluations (delivered ~ week 13)

2008 2009

Pre-survey mean (SD) N=460

Post-survey mean (SD) N=248

Cohen’s d

Pre-survey mean (SD) N=604

Post-survey mean (SD) N=321

Cohen’s d

I prefer to work on an assessment task alone*

3.48 (1.06) 3.59 (0.96) 0.11 3.63 (0.94) 3.70 (1.05) 0.07

*Likert Scale: 5 = Strongly Agree; 4 = Agree; 3 = Neutral; 2 = Disagree; 1 = Strongly Disagree

When asked about the contribution of the group assessment task to learning, student data revealed a mean improvement of 0.51 from 2008 to 2009 (Table 2). The Cohen’s effect size analysis showed a


69

small change (d = 0.42) at the high end of the spectrum. However, student confidence in their communication of chemistry concepts as a result of collaborative work is largely unchanged. Table 2: 2008 and 2009 data from end of semester course evaluation

2008

Mean (SD) N=433

2009 Mean (SD) N=323

Cohen’sd

How much did the Collaborative Task help your learning?* 2.21 (1.17) 2.72 (1.25) 0.42

As a result of the work you did in this class, what gains did you make in communicating chemical concepts to your peers.**

3.32 (0.96) 3.28 (0.95) - 0.04

*Likert Scale: 5 = Great Help; 4 = Much Help; 3 = Moderate Help; 2 = Some Help; 1 = No Help **Likert Scale: 5 = Great Gain; 4 = Good Gain; 3 = Moderate Gain; 2 = A Little Gain; 1 = No Gain

The impact of group work on student perceptions of their gains in skills related to these experiences (Table 3) had a higher mean related to affective characteristics (group building, attitude, social communication and scientific communication skills) than cognitive gains (relevance, problem solving). Table 3: Survey items developed to explore group function characteristics in 2009

Thinking about your involvement in group work, how much would you agree or disagree with the following statements?*

N Mean (Std Dev)

I have developed a positive attitude to group work 325 3.33 (1.04) As a result of group work, I have improved on my group building skills 324 3.49 (0.96) As a result of group work, I have improved on my scientific communication skills 325 3.42 (0.95) As a result of group work, I have improved on my social communication skills 325 3.35 (0.98) As a result of group work, I have improved on my problem solving skills 325 3.29 (0.96) The collaborative research task has helped me to identify the relevance of chemistry to my discipline area

324 3.11 (1.13)

To what extent did your previous chemistry studies prepare you for working with other students collaboratively

314 3.34 (1.13)

*Likert Scale: 5 = Strongly Agree; 4 = Agree; 3 = Neutral; 2 = Disagree; 1 = Strongly Disagree

DISCUSSION Collaborative student-centred learning activities offer instructors an effective strategy to address issues in diversity in learning and interests amongst students in their courses. This pedagogical approach is strongly supported by literature studies (Felder & Brent, 2001; Kagan, 1992; Johnson et al., 1998; Smith et al., 2005). It is important, however, to recognise that student attitudes and perceptions are dependent on their learning context. The present study, based in the context of a large 1st year cohort at a single institution, is part of an on-going research project to enhance student engagement and represents a focussed exploration of the role of group formation in collaborative learning. It is acknowledged that, while group formation is an essential aspect of collaborative learning, it is only one aspect of the complex, dynamic processes that characterise group work. The principal group formation variable explored in this study was the transition from random group formation of all students to the formation of structured groups either within single programs (eg BSc, Pharmacy, Biomedical Science) or in professionally aligned clusters of programs (eg Medicine & Dentistry, Business & IT). This decision was guided by the belief that collaborative communication and discourse would be enhanced in groups of students who shared common career paths. Attitudes towards group work were explored by asking students to reflect on whether they preferred to work individually on assessment tasks before and after their collaborative task. The experience of working in a group on an assessment task did not appear to have a significant impact on student preferences (Table 1) with a minor positive shift in the means observed. In exploring the impact of the collaborative task on students’ perceptions of their learning, there was a small positive shift in 2009 relative to 2008 (Table 2). The cross-correlation of the quantitative and qualitative data (Figure 1) also revealed a positive shift in attitude to collaborative work, with the number of references to group function as a positive impact on learning gains increasing in the year that structured group formation was implemented. In contrast, increased confidence in communicating


70

chemical concepts to their peers was not reflected by students. This outcome was unexpected given the intentional alignment of the topics to students’ career paths through grouping by program of study in 2009. Further exploration of the relationship between perceived relevance of the topics and learning gains is underway. Van den Bossche et al. (2001) reported that the success of collaborative work relies on two primary perspectives, cognitive and social. Overall, the combination of quantitative and qualitative data in this study revealed that the impact of group formation could be classed in one of two domains: affective factors (relating to reliance on contribution and attitudes of other individuals) and cognitive factors (learning gains, academic outcomes and perceived relevance). Comparison of the means of student perceptions of their gains in skills related to collaborative work also mirrored these two domains with gains in affective characteristics (group building, attitude, social communication and scientific communication skills) rating higher than cognitive gains (relevance, problem solving) (Table 3). Parallel insights into group work have been identified previously where negative perceptions about working with others differed to positive perceptions of group skill gains (Phipps et al., 2001). Many literature studies recommend the need to set the expectations of collaborative work and provide supporting resources for students. In 2009, students were explicitly informed of the link between skills gained by working in a collaborative environment and the professional expectations of employers. This information was supported by data gathered through a survey of employers of chemistry graduates earlier that year. 90% of employers (N=21) indicated that they highly regarded the attribute of working in groups/teams as a professional skill. Overall, evidence was found that structured group formation had a positive impact on student attitudes and learning outcomes and important insights were gained into student perceptions of the factors that influenced their learning gains. The data revealed that a criteria-based group formation approach, as opposed to a random assigned approach, had more of an impact on student learning outcomes from collaborative assessment tasks than it did on student attitudes towards working in groups for the same tasks. Student attitudes and perceptions supported published reports that socio-cultural factors strongly influence their learning gains in collaborative work. Effective group formation is not simply deliberate dispersal of students amongst groups. The balance between social and cognitive factors has emerged as important and, in the 2010 iteration of the collaborative task, students have been given the option to select both their group members and the topic that they research. It is evident that through a mixed methods evaluation and the integration of quantitative and qualitative data, greater insights into the role of group function in collaborative learning gains can be revealed.

ACKNOWLEDGEMENTS Chantal Bailey’s contribution to the statistical analysis is gratefully acknowledged. We also must acknowledge the full ALTC project team, Peter Adams, Gabriela Weaver, Lydia Kavanagh and Phil Long.This study was funded by an Australian Learning and Teaching Council Competitive Grant and a large UQ Strategic Teaching and Learning Grant. Any opinions, findings, and conclusions or recommendations expressed are those of the authors and do not necessarily reflect the views of the ALTC

REFERENCES Cohen, J. (1992) A power primer. Psychol. Bull. 112, 155–159. Coll, R.K. and Chapman, R. (2000) Choices of methodology for cooperative education researchers. Asia-Pacific J. Coop. Ed. 1

1-8. Felder, R. And Brent, R. (2001) Effective strategies for cooperative learning. J. Coop. & Collab. In CollegeTeaching 10 69-75. Gillespie, D., Rosamond, S., Thomas, E. (2006) Grouped Out? Undergraduates’ default strategies for participating in multiple

small groups. J. Gen. Ed. 55 81-102. Johnson, D.W., Johnson, R.T., and Smith, K.A. (1998) Active learning: cooperation in the college classroom. Edina, MN:

Interaction Book Co. Kagan, S (1992) Cooperative Learning. San Juan Capistrano, CA: Resources for teachers. Kavanagh, L. and Crosthwaite, C. (2007) Triple objective mentoring: achieving learning objectives with chemical engineering

students, IChemE Trans D, 2 68-79. Kriflik, L. And Mullan, J. (2007) Strategies to improve student reaction to group work. J.Teaching & Learning Prac. 4 13-27. Kuh, GD (2003) What we’re learning about student engagement from NSSE. Change 35 24-32. Phipps, M, Phipps, C, Kask, S and Higgins, S (2001) University students’ perceptions of cooperative learning: Implications for

administrators and instructors. J. Experiential Ed. 24 14-21. Prince, M and Felder, R (2006) Inductive teaching and learning methods: definitions, comparisons and research bases. J. Engr.

Ed. 95 123-38.


71

SALG (2010) Student Assessment of their Learning Gains Instrument, http://www.salgsite.org/ Last accessed June 15th 2010. Seymour, E., Wiese, D., Hunter, A. and Daffinrud, S.M. (2000). Creating a better mousetrap: On-line student assessment of

their learning gains. Paper presented at the National Meeting of the American Chemical Society, San Francisco, CA. Smith, K.A., Sheppard, S.D., Johnson, D.W. and Johnson, R.T. (2005) Pedagogies of engagement: Classroom-based

practices. J. Engr. Ed. 94 87-101. Summers, J.J., Beretvas, S.N., Svinicki, M.D., and Gorin, J.S. (2005) Evaluating collaborative learning community. J.

Experiential Ed. 73 165-88. Thompson, B. (1998) Five Methodology Errors in Educational Research: The Pantheon of Statistical Significance and Other

Faux Pas. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA, April 13-17. Assessed 22 August at http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED419023

Van den Bossche, P., Gijselaers, W.H., Segers, M., and Kirschner, P.A. (2006) Social and cognitive factors driving team learning beliefs and behaviours. Small Group Res. 37 490-521.


72

TEACHING INDUCTIVELY: GAMES IN THE TERTIARY CLASSROOM David Low ([email protected]) School of Physical, Environmental and Mathematical Sciences, The University of New South Wales at the Australian Defence Force Academy, Canberra, ACT 2600, Australia KEYWORDS: Games, curriculum, inductive teaching, learning styles

ABSTRACT Inductive teaching methods, where students construct models rather than being told facts, are aids to deeper learning, but are notoriously difficult to incorporate into mainstream tertiary teaching. Games present a relatively painless path to engage and motivate students to actively participate in the learning process. This paper presents a brief reflection on the background of, and motivation for, using games as tools for inductive teaching. The intent here is to provide examples of how games can be incorporated into a curriculum, along with some commentary on the challenges which may be encountered during development and implementation, based on the experiences of the author. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, 72-78, ISBN Number 978-0-9808597-0-6

INTRODUCTION

MOTIVATION Tertiary educators are faced by a multitude of conflicting requirements. Curricula demand both breadth and depth: the more material you elect to cover, the less you can explore each topic. Small classes are generally regarded as better learning environments, but institutions often see large lecture classes as a more efficient use of their staff. Most students would appreciate an “easy course”, while academics have a professional duty to rigorous testing of competency. No single solution has been presented to this multivariate problem: it is unlikely that one exists! However, it is undeniable that interested, motivated students are more likely to get something out of a course than disinterested, unmotivated students. Thus, a significant amount of educational research effort is spent on motivational issues, in addition to attempts to understand cognitive learning. Games have been played since time immemorial, for a variety of purposes. Martin Gardner (b.1914-d.2010) wrote 311 instalments of “Mathematical Games”, arguably the best-known section of Scientific American, over 25 years from the mid-1950’s through to the early-1980’s. The long-standing appeal of this column is reflected in the extensive set of books into which it was later collated and republished. Games have been discussed in the teaching and learning literature for at least forty years (see, e.g., Avendon & Sutton-Smith (1971), or Ellington, Addinall & Percival (1981)). Nevertheless, the use of games in formal teaching activities has been concentrated at the primary school level (see, e.g., Ellington, Fowlie & Gordon (1998), or Cruickshank & Telfer (1980)). Furthermore, as discussed by Selkirk (1986), games have been seen as primarily supporting the mathematics curriculum. A notable exception to both points would be the exploratory roleplaying activities implemented by Francis and Byrne (1999), which involve students taking on the part of scientists of varying specialities, working collectively to understand the universe (and gain professional kudos at the same time!). Classroom games at the tertiary level are likely better suited to developing science-based graduate attributes, rather than assisting students learning the specifics of any discipline (barring, perhaps, algorithmic games used as computer science/programming exercises). It is the aim of this work to demonstrate some examples of both types, where games can be used in the tertiary classroom to motivate students, encourage their active involvement, and develop understanding. While “general science” courses have been out of favour in Australian universities for many years, there are some signs (such as Australia actively engaging with the Bologna Process as of April 2009) that the wheel may be turning, in which case one may expect a greater emphasis on generalist critical-thinking in tertiary education.

Low, Teaching Inductively: Games in the Tertiary Classroom

73

INDUCTIVE AND DEDUCTIVE APPROACHES TO TEACHING AND LEARNING As children, we learn how the world works by an inductive process: we try something, observe the result, and draw inferences. If you don’t know how something works, this is a natural approach – and, of course, it strongly resembles the “testing” part of the conventional scientific method (which includes preceding steps of “observe” and “hypothesise”, and a subsequent “reform” step). Given that a teacher already understands the material, however, conventional teaching approaches – especially at the tertiary level – tend more towards a deductive process: starting from a general principle, derive implications. Induction is messy, but promotes deeper learning; deduction allows for control and pacing of material, but has a tendency towards learning by mimicry (Felder & Silverman, 1988). It is interesting – and perhaps a little disappointing – that teachers often prefer to use a deductive style because it allows control over presentation; and that students often prefer deductive teaching because it is neat and compartmentalised, and conforms to their expectations and experiences from prior classroom teaching. It takes effort (and, perhaps, courage) for both teachers and students to engage in an inductive style of learning. In reality, given constraints such as the need to cover a syllabus, the inductive style of teaching is probably best suited to cases where the underlying “rule of nature” can be relatively easily determined. As practising scientists are well aware, a great deal of time, and a large amount of frustration, is usually involved in attempts to understand natural processes! Inductive teaching can be implemented in a number of ways, under a number of names. Prince and Felder (2006, 2007) present an extensive comparison of inductive methods in common use, such as inquiry-based learning, problem-based learning, project-based learning, case-based teaching, discovery learning, and just-in-time teaching. The common aim is to start from specifics (a set of observations, or a particular problem), and encourage and guide students to discover the underlying principles for themselves: “learning by doing”, rather than “teaching by telling”. Games, we propose, can be used in a similar way: encouraging students to become actively involved in the classroom, by temporarily removing them from it (in an intellectual sense!).

EXAMPLES In this section, we consider two quite distinct types of games which have been used by the author in different classroom settings. The aim here is to summarise the rules of each game, and then to discuss the implications for teaching and learning. For further details on the specifics of each game, the reader should consult the primary references directly.

THE SCIENTIFIC METHOD: ELEUSIS AND ZENDO Robert Abbott invented the game Eleusis in 1956, and it was publicised by Gardner (1959, 1977). Zendo (Heath, 1997) is a modern development, with the most significant difference being that Eleusis uses a standard deck of playing cards and focuses on mathematical and colour/suit sequences, while Zendo uses abstract shapes and arrangements. A similar activity, based on the game Patterns (Sackson, 1969), has been developed into a classroom activity by McCoy (1999). The games vary in details such as scoring, yet have a common theme: 1. the moderator (“Nature”) decides on a hidden law, which the players must try to determine. For

example, “always change suits on subsequent plays”, or “no more than two pieces of the same shape”. The moderator then displays two sequences or arrangements, one that obeys the law and one that does not;

2. players (“scientists”) take turns (“perform experiments”) by constructing their own sequences/arrangements for the moderator to evaluate. The moderator declares each attempt as being either consistent with the hidden law, or contrary to it;

3. if a player thinks they have determined the hidden law, they elucidate it (“make a hypothesis”). If it is correct, the game ends; if it is incorrect, the moderator shows a counter-example which is consistent with the hidden law, yet contrary to the stated hypothesis, and the game continues.

With a skilled moderator, adept at misdirection in the examples and counter-examples, even simple laws – the only ones which are recommended in a teaching environment – can be a challenge to determine. With repeated play, students learn a few things about science:


74

1. the results of simple experiments are easier to interpret than those of complicated ones (Occam’s Razor);

2. a control experiment, followed by a series of small variations to a minimal number of parameters, can help identify the key aspects of a problem; and

3. counter-examples are something to be embraced, rather than feared! Preconceptions based on spurious correlations often require a counter-example to force a change of thinking.

When used as a teaching tool rather than simply as entertainment, it is important to include a reflection stage once the game has concluded: students should be encouraged to discuss their personal choices, how the “experiments” of other students affected their own play, and whether there were any “Eureka!” moments; or, as is perhaps more often the case in real science, if there were any, “That’s not what I expected…” moments. It is equally important to point out the challenges that face scientists in reality, which the game does not model: philosophically speaking, nature does not always throw up counter-examples on demand; prosaically, experiments cost money and take time; and critically, the “real” game doesn’t end with a declaration of truth! Educators may wish to avoid issues such as the kudos of publishing positive results introducing a bias against spending a scientific career obtaining negative results, although it may lead to a discussion of ethics which is often missing from science degrees! At UNSW@ADFA, this game has been used with small numbers (1-4) of first-year science students in the Chief of Defence Force Scholars Program (CDFSP), who undertake a “research oriented” Bachelor of Science degree where 1/6th of their coursework program is replaced by research training and project work. While small groups are important for the game to avoid significant downtime (even simultaneous turns by players require evaluation time from the moderator), any science course, at any level, could benefit from the activity.

NEWTON-I, VECTORS, FRICTION AND GRAVITATION: RACETRACK AND TRIPLANETARY The origins of the pencil-and-paper game Racetrack are unclear, but it was first drawn to public attention by Gardner (1973), with further details given in a republished collection (Gardner, 1986). Commercial versions include the boardgames Tacara (pub. Eggert-Spiele, 2000) and Bolide (pub. Rio Grande Games, 2005). Fundamentally, it is a simulation game, modeling cars racing around a circuit, using simple Newtonian kinematics. The game is played on a sheet of square-gridded graph paper, using the intersections of the lines. To prepare for play, a closed track should be drawn on the gridded paper: the track width can vary from a few to a dozen or more squares; and better games result on tracks with varying degrees of curvature. Mark a start/finish line, upon which each player marks their car’s starting location. Turns consist of players moving their car as follows:

1. firstly, as an intermediate step, duplicate the player’s move of the previous turn (for the first turn, treat this is a null move);

2. then, if desired, alter the final destination for this turn by one grid point in any direction; 3. draw a line connecting the previous and new destination points.

See Figure 1 for an example of how turns are constructed. In effect, from turn to turn, a car can either maintain its velocity, or change it by one unit in any direction. The aim of the game is to complete the circuit in the least number of moves, without the car’s path leaving the track, or being in the same place as another car at the same time. The game is well-suited to attempted solution by algorithmic means, and presents a programming challenge to students of computer science (see, for example, Holzer & McKenzie, (2010)).Figure 2 shows an example of a completed game.


75

Figure 1: An example of how turns are constructed in Racetrack. Image sourced from http://www.boardgamegeek.com/file/download/4gzlxve9vs/Drifter_Race.pdf There are some obvious ways that Racetrack can be used to illustrate vectors: for example, ideas such as v2 = v1 + ∆v are clear, but the often confusing equivalence ∆v = v2 – v1 is also simple to demonstrate graphically. This can be tied to the concept and mathematics of acceleration, if each turn is given a fixed time duration. It is equally obvious that the movement system is based on Newton’s First Law (N1); but rather than stating this and showing how the game implements it (a deductive approach), the game can be introduced before Newton’s Laws are discussed, and thus used as an inductive learning tool. This can work well in a large first-year physics class, and has been conducted with n ≈ 200 students in a tiered theatre at UNSW@ADFA. If students are provided with a pre-printed track, and instructed in the rules of the game, it is usually not difficult to persuade them to participate in groups of two or three. After a few minutes of play, regain the attention of the class (often the most difficult part of the exercise…) and explain that the lead driver has encountered an oil slick (or ice) on the road, extending from the current location of the lead car to a line some distance ahead. Then, by asking the students to discuss how the rules should be modified to deal with this situation, ideas such as N1 can be introduced by the students, rather than by the teacher. With guidance, the discussion can be extended to other concepts, including acceleration as the rate of change of velocity, the role of friction between tyres and road during acceleration (both in magnitude and direction). The game can be reintroduced in later classes when, for example, one wishes to discuss forces, acceleration and centripetal motion (addressing questions such as, “what is happening as a car goes around a corner?”). If students discover clues for themselves via the game, they are far more likely to recall and accept them, as part of their internal world-view, than if they are simply told how things “are”. Variations of this game, featuring spacecraft navigating through the solar system (albeit in two dimensions), have been described by Vinson (1998) and Lowry (2008). The former is a direct translation of Racetrack to a spacecraft theme; the latter is notable for including a mathematical implementation of inverse-square gravitation, allowing for direct comparison of the effect of gravity on objects due to different mass stellar objects. Intermediate in complexity, and with some advantages for classroom use, is the 2D-astrogation aspect of the game Triplanetary (Miller & Harshman, 1973).

← Car starts here, with a velocity of “two squares down” ← First move to here, with no change in velocity.

← Default for the next move is to position X (at end of dashed line), but there are options, as marked with small circles. ← The driver chooses to move here (front-right option). ← Again, the dashed line terminating at X shows the default next move (duplicating the previous move). The driver chooses the back-left option. ← The new default path is once again “two squares down”, as that duplicates the previous move; options for the next move are shown with larger circles.


76

Figure 2: An example of a completed game of Racetrack, including commentary. Image source http://www.boardgamegeek.com/image/114849/racetrack. Rather than playing on the intersections of a square grid, Triplanetary uses the spaces on a hexagonal grid. Hexagons are the regular polygon which, when tessellated, allows the greatest degree of directional flexibility, permitting equiscaled motion in six directions, and avoiding the √2 scaling issue which is tacitly ignored during diagonal movement in Racetrack. As well as providing a “friction free” environment, and thus perhaps a cleaner picture of how velocity is changed by expending fuel, Triplanetary provides an excellent forum for the discussion of relative velocity (“From your point of view, what does it look like that other spaceship is doing?”). A multi-part exercise can also serve as an introduction to gravitation, by asking students to navigate (“astrogate”?) around a stylised solar system. The first time the exercise is offered, it can be with the (unrealistic) situation of unlimited fuel for velocity changes, and ask students to minimise their fuel usage. This can be followed by discussion of how the simple movement model could be changed to account for the gravity of large bodies such as planets: in effect, asking students to come up with a rule for gravity! One interesting way to introduce this idea is by considering the Earth-Moon system and asking what forces/accelerations must be acting to keep the Moon in orbit. This process, of attempting to determine (how to modify) the “rules of the game” are yet another example of the inductive learning process in action. Once the students have a (reasonable!) gravity model/rule in place, the astrogation exercise can be re-run (usually as a take-home activity), with the students attempting to minimize their fuel usage by taking advantage of gravity-assisted “slingshot” maneuvers. Followup discussion can mention the variety of methods used to guide space probes in real-life exploration of the solar system. Figure 3 shows how Triplanetary deals with gravitation in 2D: it is interesting to note that under this rule-set, planetary orbits can be entered naturally, with no need for additional rules!


77

Figure 3: The implementation of gravitation in Triplanetary, sourced from the Triplanetary rulebook, http://www.boardgamegeek.com/file/download/ymglb856x/Triplanetary.pdf

While three-dimensional Newtonian vector movement games exist (e.g. Ad Astra Games publishes Attack Vector Tactical (2004), a full 3D space game, and Birds of Prey (2008), simulating jet-fighter air-to-air combat), they are arguably too complicated (in that they become too realistic!) to include in a classroom setting. Indeed, a reasonable guideline to the use of games in the classroom is that the game engine should not be more complicated than the process it is trying to model, to avoid the possibility that the details of the implementation overwhelm the message. The purpose of using games as an inductive learning tool is primarily to motivate and engage the students, and highlight the processes involved by allowing the students to discover them for themselves. Moving along the path from “game” to “simulation” is similar to moving from an inductive to a deductive process.

DISCUSSION Objective evaluation of the efficacy of these methods is difficult, as there is no control sample against which to compare. In addition, for any particular course, these approaches were used as a small part of a larger course, rather than as the dominant technique. However, students were regularly asked to comment on the variety of teaching methods used in these courses. Responses of a positive nature strongly featured keywords such as “interactive”, “active” and “conceptual” when referring to the classroom activities and the development of their understanding. For example:

- “Interactive lessons and demonstrations that explained and entertained” - “Involved the class via quirky examples” - “Very interactive teaching encouraged discussion” - “Engaging: only lectures where I could stay awake – and focus” - “Conceptual understanding as the basis of the course”

On the negative side, as early as halfway through their first semester of tertiary study, students in a large first-year class (n ≈ 200) were well aware that the time spent with these activities took time away from other things, such as numerical worked examples, theoretical derivations, and depth of coverage. A selection of comments relating to the “ways in which the teaching could be improved” from this class included:

- “Difficult to relate teaching to practical problem solving” - “Would prefer a factual rather than conceptual approach” - “Teach us! Too much work to do by self-study”

A spacecraft at A moves to B under its pre-existing velocity. In the absence of external forces, it would continue to move to C next turn. Triplanetary models gravity with “gravity arrows” surrounding a planetary body (the filled circle to the lower left of hex B). If a move crosses through a gravity arrow, then the next move must include one-hex accelerations in that direction. Hence, instead of finishing at C, the spacecraft experiences an acceleration CD (due to the gravity arrow between A and B), plus an acceleration DE (due to the gravity arrow at B), and actually follows path BE.

On the next move, in the absence of external forces, the move would be EF. However, the gravity arrow traversed in the last move (just after hex B) causes an acceleration FG, and actual path EG.


78

- “Focus on the specific content…to allow the class time to understand the concept before presenting thought-provoking material to further enhance learning”

While this dichotomy could be explained by students being accustomed to classwork being a mix of theory plus numerical problems, it is also clear that, in a large first-year class for example, there will be a wide mix of preferred learning styles (Low, 2009), as well as an understandable reticence for students to embrace being taken out of their comfort zone. It is clearly important for inductive teaching methods to be used carefully, for students to be briefed on why certain approaches are being used, and – perhaps most importantly to students – how they will tie in with assessment. These cautions are, naturally enough, common to any formal educational activity. Nevertheless, if one accepts that active students are more likely to learn than passive students, and if the development of “active minds” is our goal, then the use of games as teaching tools in the tertiary environment certainly presents a pathway to that end. Tertiary education has not yet reached the stage where every activity requires a predetermined outcome, with a measurable result, and an associated evaluation metric: thankfully, in the author’s opinion, we still have the freedom to give students the opportunity to play.

REFERENCES Avendon, E. & Sutton-Smith, B. (Eds) (1971) The study of games. New York: John Wiley & Sons. Cruickshank, D. R. & Telfer, R. (1980) Classroom games and simulations. Theory into Practice, 19(1), 75-80. Ellington, H., Addinall, E. & Percival, F. (1981) Games and simulations in science education. London: Kogan Page Ltd. Ellington, H., Fowlie, J. & Gordon, M. (1998) Using games and simulations in the classroom: a practical guide for teachers.

Routledge. Felder, R. M. & Silverman, L. K. (1988) Learning and teaching styles in engineering education. J. Engr. Educ., 78(7), 674-681.

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.92.774&rep=rep1&type=pdf Francis, P. J. & Byrne, A. P. (1999) Use of role-playing exercises in teaching undergraduate astronomy and physics. Publ.

Astron. Soc. Aust., 16(2), 206-211. http://www.atnf.csiro.au/pasa/16_2/francis/paper.pdf Gardner, M. (1959) Mathematical Games. Scientific American, 200(6), 160-168. Gardner, M. (1973) Mathematical Games. Scientific American, 228(1), 108-115. Gardner, M. (1977) Mathematical Games. Scientific American, 237(4), 18-25. Gardner, M. (1986) Knotted doughnuts and other mathematical entertainments. New York: W. H. Freeman and Company. Heath, K. (1997) Zendo. College Park, MD: Looney Labs. Holzer, M. & McKenzie, P. (2010) The computational complexity of RACETRACK. Proceedings of 5th International

Conference, Fun with Algorithms (FUN 2010), 2nd-4th June 2010, Iscia, Italy. Low, D. (2009) Student perceptions of lecture approaches in first-year Engineering Physics. Proceedings of the 2009 UniServe

Science Conference, 1st-2nd October, Sydney, Australia. http://sydney.edu.au/science/uniserve_science/images/content/2009_papers/low.pdf

Lowry, M. (2008) Teaching universal gravitation with vector games. The Physics Teacher, 46(9), 519-521. McCoy, R. (1999) Inductive reasoning: the game of patterns. Project PHYSNET. East Lansing, MI: Michigan State University.

http://physnet2.pa.msu.edu/home/modules/pdf_modules/m70.pdf Miller, M. W. & Harshman, J. (1973) Triplanetary. Normal, IL: Game Designers’ Workshop. Prince, M. & Felder, R. (2006) Inductive teaching and learning methods: definitions, comparisons, and research bases. J. Engr.

Education, 95(2), 123-138. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.62.8102&rep=rep1&type=pdf Prince, M. & Felder, R. (2007) The many faces of inductive teaching and learning. J. College Sci. Teaching, 36(5), 14-20.

http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/Inductive(JCST).pdf Sackson, S. (1969) A gamut of games. Random House. Selkirk, K. (1986) Simulation games: a mathematical activity. Mathematics in Schools, 15(1), 40-43.

http://www.jstor.org/stable/30216311 Vinson, M. (1998) Space Race: a game of physics adventure. The Physics Teacher, 36(1), 20-21.

Muir, Ryan, Drury, Feedback in the Sciences: What is Wanted and What is Given

79

FEEDBACK IN THE SCIENCES: WHAT IS WANTED AND WHAT IS GIVEN Meloni M. Muira, Lorraine M. Ryanb, Helen Druryb Presenting author: Meloni Muir ([email protected]) a Faculty of Medicine, University of Sydney, Sydney NSW 2006, Australia b Learning Centre, University of Sydney, Sydney NSW 2006, Australia

KEYWORDS: feedback, student perspective, student learning, feedback model

ABSTRACT This paper reports the initial findings from a study investigating students’ perceptions of the feedback they have received at university in order to gain insight into the kinds of feedback most sought by students. Four hundred and nineteen second-year university Science students completed a 53 item questionnaire. Using a Likert scale, the questionnaire examined i) how students experienced feedback, ii) what students did with feedback, iii) how useful students perceived feedback to be, and iv) what type of written feedback was important to students. Statistical analyses of the data indicated that students carefully read feedback, used it to both go over the current assignment and improve future assignments, and that feedback received contributed to their understanding of course content. In addition, the data showed that a significant majority of students reported both positive and negative feedback as useful. The results suggest that students use written feedback not only for reflection on the assessment for which it was provided but to feed forward on future assessments. The results will be discussed in relation to the model of feedback proposed by Hattie and Timperley (2007). Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 79-84, ISBN Number 978-0-9808597-0-6

INTRODUCTION Giving feedback to students on their written assignments has long been accepted as essential practice in university teaching (Biggs, 1999; Gibbs, 1999; Hounsell, 1987; Ramsden, 1992). There is substantial evidence that feedback can have a powerful influence on student learning and achievement compared with other aspects of teaching (Black & William, 1998; Hattie, 1987, 1999). Feedback on written assignments can be seen as part of the communication and negotiation process between students and their lecturers in which students are apprenticed into the discourse community of their discipline (Swales, 1990). It is through this developmental process that students come to share their lecturer’s understandings of the kind of written communication valued by the discipline (Laurillard, 2002; Sadler, 1989, 1998). Research into students’ perspectives on feedback on their written assignments has shown overwhelmingly that feedback - independent of quality or quantity - is highly valued by students (Hartley & Skelton, 2002; Higgins, Orsmond, Merry & Reiling, 2005; Hyland, 2000; O’Donovan, Price & Rust, 2001). The provision of effective feedback, however, is subject to a number of constraints and presents substantial challenges. It has been suggested that students may not read feedback provided (Hounsell, 1987) or, if they do, they may not understand it or use it (Gibbs & Simpson, 2004; Lea & Street, 1998; McCune, 2004). In addition, in a study by MacLellan (2001), most students indicated feedback was only somewhat helpful in their understanding and learning, with nearly a third reporting that feedback was never helpful. In other published studies, students have reported feedback to be too vague or too subjective (Holmes & Smith, 2003). From a staff perspective, providing feedback is time consuming and to be effective, its delivery is time dependent. With increasing student numbers and resource constraints, in many universities there has been an increase in the use of sessional staff, as well as a reduction in the frequency of assessments, the quality and quantity of feedback and its timely provision (Gibbs & Simpson, 2004). Also, as university courses move towards a modularised semester system, feedback may only occur when assignments are returned towards the end of a semester allowing for little, if any, formative feedback. Providing written feedback on student report writing in the sciences is an integral part of the teaching-learning cycle. It is essential that to enhance student learning, feedback must be effective. In their model of feedback, Hattie and Timperley (2007) propose that “effective feedback must answer three major questions asked by a teacher and/or by a student: Where am I going? (What are the goals?),


80

How am I going? (What progress is being made toward the goal?) and Where to next? (What activities need to be undertaken to make better progress?).” The results presented in this paper are part of a larger investigation into how Science students use feedback in their written assessments and the kinds of feedback most sought, with an aim to developing workshops to assists markers in improving their provision of effective feedback. In this paper, the results will be discussed in relation to Hattie and Timperley’s (2007) model.

METHODS A seven-section questionnaire (sections A-G) was administered to 419 second-year university Science students. The student sample was 62.6% female with a mean age of 20.2 years (range 18-47 years). Questionnaires were completed by students during class. Students were instructed to respond to questionnaire items based on their experience at university in general, not simply in their current unit of study. Sections A and B of the questionnaire focused on students’ language background and tertiary writing history. Section C consisted of open-ended questions regarding their perceptions of feedback. The remaining four sections, D-G, consisted of a total of 53 items relating to i) students’ perceptions of quantity, timing and quality of feedback (section D), ii) students’ attitudes towards feedback (section D), iii) how students used and felt about feedback received (section F), and (iv) what types of written feedback were important (section G). Responses were recorded on a 4-point Likert scale indicating strength and direction of endorsement. To simplify analyses, responses were collapsed to form two categories corresponding to endorsement or rejection of the item; responses corresponding to 1 and 2 on the scale were grouped into “disagree” (or for section G, “not important”) and those corresponding to 3 and 4 were grouped into “agree” (or for section G “important”). The number of participants in each category was then compared using Chi Square goodness of fit analyses.

RESULTS Table 1 summarises students’ attitudes regarding the feedback received. Most participants indicated that the written feedback they received was related to the assessment criteria, course objectives and to the marks given but was too brief. Most students considered negative feedback to be constructive and did not ignore it (Table 2) or report negative reactions, such as feeling demoralised or angry, in response to it. Positive feedback was reported to boost confidence (Table 2). The only statistically non-significant result in Table 1 was in relation to verbal feedback. Students were almost equally divided on whether they remembered verbal feedback, although a significant majority indicated that they received verbal feedback. Table 1: Number of participants’ endorsing/rejecting section D items and corresponding chi square analyses

Section D: How do you experience feedback? Disagree Agree

2

1.I receive verbal feedback from teaching staff on my assignment(s) 312 95 115.70**

2.I receive written feedback from teaching staff on my assignment(s) 151 256 27.09**

3.I forget verbal feedback (on my assignments) easily 197 210 0.42 4.I feel demoralised or angry after reading negative feedback 364 43 253.17** 5.I think about giving up when I get negative feedback 357 50 231.57** 6.I see negative feedback as constructive 41 367 260.48** 7.Feedback, when handwritten, is easy to read 131 271 48.76** 8.Written feedback is related to the mark I get 95 311 114.92** 9.Written feedback is related to assessment criteria 73 332 165.63** 10.Written feedback is related to course or unit of study objectives 118 284 68.55** 11.Written feedback is too brief 118 286 69.86**

* 2 significant at .05 level; **

2 significant at .01 level


81

With regard to how students used feedback (Table 2), the majority reported carefully reading and using feedback to go over the current assignment or revise work, as well as to improve future assignments. More than two-thirds of the students used the feedback even if a high grade was achieved. Students reported that receiving only a mark was unhelpful. All of these results were statistically significant. Although a significant majority of students found markers to be consistent in their application of assessment criteria, there was sizeable non-significant minority who did not find markers consistent. Table 2: Number of participants’ endorsing/rejecting section E items and corresponding chi square analyses

Section E: What do you do with feedback? Disagree Agree 2

1.I read feedback carefully and try to understand what the feedback is saying 24 384 317.65**

2.I use feedback to go over what I have done in the assignment 51 356 228.56**

3.I act on feedback suggestions to improve my future assignments 31 377 293.42**

4.I have good intentions but forget feedback suggestions for improvement on my future assignments

249 160 19.37**

5.I use feedback only when I get a low grade 273 133 48.28**

6.I tend to read only the marks 336 71 172.54**

7.I do not use feedback for revising 293 115 77.66**

8.Positive feedback boosts my confidence 26 383 311.61**

9.I ignore negative feedback 386 21 327.33**

10.Feedback that tells me ONLY my grade does not help me 64 341 189.45**

11.I can’t learn from feedback because markers differ in the way they apply the assessment criteria when marking my assignments

265 143 36.48**



Table 3: Number of participants’ endorsing/rejecting section F items and corresponding chi square analyses

Section F: How useful do you find feedback? Disagree Agree 2

1.Feedback mainly tells me how well I am doing in relation to others 250 160 19.76**

2.Feedback is helpful to explain gaps in my knowledge and understanding 51 361 233.25**

3.Feedback provides me with useful suggestions for improvement in my assignments 47 365 245.45**

4.Feedback helps me to improve my ways of learning and studying 89 322 132.09**

5.Feedback helps me to reflect on what I have learned 130 280 54.88**

6.Once I have read the feedback I understand why I got the mark I did 130 281 55.48**

7.Feedback on assignments given to the whole class helps me to learn 185 223 3.54

8.Feedback to the whole class helps me to understand what I did right and wrong in my assignment 191 219 1.91

9.Individual feedback helps me to understand what I did right and wrong in my assignment 16 394 348.50**

10.I receive feedback on my assignment(s) in time for it to be useful for the next assignment 229 180 5.87*

11.Feedback does not help me with future assignments 349 58 208.06**

12.Feedback prompts me to go back over material covered earlier in the course 171 238 10.98**

13.Feedback encourages me to improve 42 367 258.25**

14.Written feedback is difficult to apply 343 67 185.80**

15.Written feedback is informative 61 348 201.39**




82

The data significantly indicated that students found feedback helpful for improving their work, identifying gaps in their knowledge and understanding the course content (Table 3). Furthermore, students indicated that feedback encouraged reflection on what they had learned, prompted revision and encouraged them to improve. Feedback given to the whole class, however, did not appear to be regarded as helpful by the majority of students, with nearly equivalent numbers finding class-based feedback helpful and unhelpful. Table 4: Number of participants’ endorsing/rejecting section G items and corresponding chi square analyses

SECTION G: What type of written feedback is important? Not Important

Important 2

1.Feedback on your written assignments that tells you what you could do to improve 10 377 348.03**

2.Feedback on your written assignments that explains your mistakes in understanding subject matter 16 369 323.66**

3.Feedback on your written assignments that corrects your mistakes in subject matter 30 356 275.33**

4.Feedback on your written assignments that explains your mistakes in your use of language 83 303 125.39**

5.Feedback on your written assignments that corrects your mistakes in using language 103 280 81.80**

6.Feedback on your written assignments that tells you what you have done badly 26 360 289.01**

7.Feedback on your written assignments that tells you what you have done well 50 334 210.04**

8.Feedback on your written assignments that provides you with general comments 190 195 .07

9.Feedback on your written assignments that focuses on the subject matter 46 339 222.98**

10.Feedback on your written assignments that focuses on how you have written critically about a topic/experiment/essay 43 343 223.16**

11.Feedback on your written assignments that focuses on how you have argued in your writing 68 317 161.04**

12.Feedback on your written assignments that focuses on your use of evidence from sources in your writing 66 318 165.38**

13.Feedback on your written assignments that explains your grade 39 346 244.80**



DISCUSSION For feedback to influence learning and student performance, teachers need to convey to students not only the gap between their performance level and the expected level but also how to move towards closing this gap. The model of feedback proposed by Hattie and Timperley (2007) addresses these issues by posing three questions: Where am I going?, How am I going? and Where to next?

WHERE AM I GOING? This question relates to goals and therefore to assessment criteria. For items D8-10, E5, 11, and G13, students strongly agreed with or reported them to be extremely important (Tables 1, 2, 4). All of these items relate to assessment criteria and explanation of the grade received. This type of feedback informs students as to their progress toward the attainment of learning goals related to particular assessments and is referred to as the “feed up” dimension (Hattie & Timperley, 2007).


83

HOW AM I GOING? This question relates to actual progress being made towards the goal, for example the assessment criteria, as well as how to proceed. This feedback is intended to promote reflection on what students have and have not learned (F2, 5) and on what is correct and incorrect in their knowledge (F9, G2-5), as well as encouraging student revision (E2, F12) and indicating their position in relation to peers (F1). Students strongly agreed with all of these items (or for section G, evaluated them as “important”; Tables 2-4). Based on the data, we suggest that students recognise the value of this type of feedback in enhancing their learning and achieving assessment goals. Hattie and Timperley (2007) refer to this aspect of feedback as “the feed-back dimension.”

WHERE TO NEXT? Hattie and Timperley (2007) suggest that this question is best answered by “providing information that leads to greater possibilities for learning.” The question relates to applying feedback received to improve future learning experiences and is referred to as the “feed forward” dimension of feedback (Hattie & Timperley, 2007). In our data, students strongly agreed that feedback was helpful in improving their learning, studying and hence performance in future assessments (E3, F3-4, F10-11, G1). Such a perception is encouraging because feed forward is arguably one of the overriding goals of teaching, empowering students to move beyond the context of the current assessment towards enhanced, self-directed learning. Effective feedback can be both positive and negative (Kluger & DeNisi, 1996). To be effective, both types need to offer students information relevant to the assessment rather than commenting on the student as person (Gibbs & Simpson, 2004; Hattie & Timperley, 2007). In our data, students strongly agreed that positive feedback boosted confidence (E8) and encouraged improvement (F13; Tables 2, 3), while negative feedback was not ignored (E9) but rather viewed as constructive (D6), and did not affect motivation (D4, 5; Tables 1, 2). These data suggest that feedback received focused on the assessment rather than the “self as a person”, thus providing information relating to the three questions posed in the Hattie and Timperley (2007) model. Feedback can have a significant impact on student learning. For feedback to influence learning and performance, teachers need to convey to students not only the gap between their performance level and the expected level but also how to move towards closing this gap. The results of our study indicate that students are getting what they want in terms of feedback, and that this feedback is relating “How am I going?” to “Where am I going?” and pointing students towards “Where to next?” A question remaining for students may be “When will I get there?”

ACKNOWLEGEMENTS The authors wish to gratefully acknowledge Miriam Frommer, Vanessa Gysbers, Sadhana Raju and Fiona White for their assistance in questionnaire development and data collection.

REFERENCES Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-74. Biggs, J. (1999). Teaching for Quality Learning at University. Buckingham: Society for Research into Higher Education & Open

University Press. Gibbs, G. (1999). Using assessment strategically to change the way students learn. In S. Brown & A. Glasner (Eds.),

Assessment Matters in Higher Education. Buckingham: Society for Research into Higher Education & Open University Press.

Gibbs, G. & Simpson, C. (2004). Conditions under which assessment supports students’ learning. Learning and Teaching in Higher Education. 1, 3-31.

Hattie, J. (1987). Identifying the salient facets of a model of student learning: a synthesis of meta-analyses, International Journal of Education Research, 11, 187-212.

Hattie, J. (1999). Influences on student learning (Inaugural professorial address, University of Auckland, NZ). http://www.education.auckland.ac.nz/uoa/home/about/staff/j.hattie/hattie-papers-download/influences

Hattie, J. & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. Higgins, R., Hartley, P. & Skelton, A. (2002). The conscientious consumer: reconsidering the role of assessment feedback in

student learning. Studies in Higher Education, 27(1), 53-64. Holmes, L. & Smith, L. (2003). Student evaluations of faculty grading methods. Journal of Education for Business, 78(6), 318-

323. Hounsell, D. (1987). Essay Writing & the Quality of Feedback. In J. Richardson, M. Eysenck & W. Piper (Eds.), Student

Learning: Research in Education & Cognitive Psychology. Milton Keynes: Open University Press and Society for Research into Higher Education.


84

Hyland, P. (2000). Learning from feedback on assessment. In P. Hyland & A. Booth (Eds.) The Practice of University History Teaching. Manchester: Manchester University Press.

Kluger, A. & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284

Laurillard, D. (2002). Rethinking university teaching: A conversational framework for the effective use of learning technologies (2nd ed.). London: Routledge Falmer.

Lea, M. & Street, B. (1998). Student writing in higher education: an academic literacies approach. Studies in Higher Education, 23(2), 157-172

MacLellan, E. (2001). Assessment for Learning: the differing perceptions of tutors and students. Assessment and Evaluation, 26(4), 307-318.

McCune, V. (2004). Development of first-year students’ conceptions of essay writing. Higher Education, 47, 257-282. O’Donovan, B., Price, M. & Rust, C. (2001). The student experience of criterion-referenced assessment. Innovation in

Education and Teaching International, 38(1), 74-85. Orsmond, P., Merry, S., & Reiling, K. (2005). Biology students' utilization of tutors' formative feedback: A qualitative interview

study. Assessment & Evaluation in Higher Education, 30(4), 369-386. Ramsden, P. (1992). Learning to Teach in Higher Education. London: Routledge. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144. Sadler, D. R. (1998). Formative assessment: Revisiting the territory. Assessment in Education, 5(1), 77-84. Swales, J. M. (1990). Genre analysis English in academic settings. Cambridge: Cambridge University Press.

O’Brien, Jarrett, Purser, Brown, Mapping Science Subjects: A Ground Up Approach

85

MAPPING SCIENCE SUBJECTS: A GROUND UP APPROACH Glennys O'Briena, Lorna Jarrettb, Emily Purserc, Christine Brownd Presenting author: Glennys O’Brien ([email protected]) a School of Chemistry, University of Wollongong, Wollongong NSW 2522, Australia b Faculty of Education, University of Wollongong, Wollongong NSW 2522, Australia c Learning Development Unit, University of Wollongong, Wollongong NSW 2522, Australia d Academic Service Division, University of Wollongong, Wollongong NSW 2522, Australia KEYWORDS: curriculum mapping; "unit of study" mapping; curriculum mapping methodology

ABSTRACT The need to clearly demonstrate the components and outcomes of a curriculum is a major factor in the drive for quality assurance manifest across the tertiary education sector. This project is a detailed gathering of commentary and data about the subjects offered in the Faculty of Science, University of Wollongong (UOW). The project aims to provide a means of tracking concept and skill development through curricula, to identify sharable resources and teaching practice, to clarify support needs and to provide a means for storing and maintaining an ongoing record of commentary and data about each subject. The investigative approach is a type of curriculum mapping based on interviews with key players in the design, delivery and reception of the curriculum. In the process all available materials and data about each subject were gathered. The methodology has been developed and used first for mapping of subjects within the School of Chemistry, providing a tested and flexible process to facilitate the investigation in the other Schools in the faculty. For Chemistry subjects a dataset of information is now available from which developments in curriculum and teaching management are proceeding. From staff and student interviews and our collective experience we can also report valuable commentary. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 85-91, ISBN Number 978-0-9808597-0-6

INTRODUCTION There is a prevailing drive for quality assurance in teaching and learning across the tertiary education sector. This is bringing to the fore development of various methodologies for the documentation and mapping of all components and outcomes of curricula. Within this context the motivation for this project was to find out as much as possible about the subjects1 offered in the Faculty of Science at the University of Wollongong from multiple points of view, investigating, developing and employing a variety of curriculum mapping and analysis techniques and procedures in the process. There is a great diversity in rationale and methods for curriculum mapping reported in the literature, depending on the focus of a particular project, the stage of curricula development and the perspectives of those involved. Curriculum mapping is carried out to follow any characteristic of a degree program through that program to ensure coherency (O’Neill, 2009). Where a program consists of various modules, especially some from different disciplines or faculties, it is essential to ensure the sum of the parts makes the desired whole in all aspects of curriculum content, delivery and assessment. Mapping is often focussed on generic or graduate attributes and/or skills (Aabakken & Bach-Gasmo, 2000; Hege, Siebeck & Fischer, 2007; Plaza, 2007; Robely, Whittle & Murdoch-Eaton, 2005; Ross, 1999; Wachtler & Troein, 2003; Willett, 2008). Mapping is also an essential first step in the process of aligning the intended, the delivered and the received curricula (Bath, Smith, Stein & Swannet, 2004; Bruinsma & Jansen, 2007; Harden, 2000; Plaza, 2007; Robely et al., 2005; Wachtler & Troein, 2003). Less widely acknowledged is the importance of explicitly collecting data from students, for example by incorporating survey data into the curriculum mapping dataset (Bath et al. 2004; Bruinsma & Jansen 2007; Plaza 2007; Wachtler & Troein, 2003), which suggests that the received curriculum may not always be taken into account. 1 Terms: (i) ‘subject' is used at UOW to designate a unit of study for which credit is gained, 'subject’ here does not refer to the topic, field, discipline or area of study; (ii) ‘course’ is used to designate the complete degree program, composed of ‘subjects;' (iii) coordinator is the academic staff member primarily responsible for the discipline content, design, teaching and delivery of a subject; (iv)‘subject outline’ is the document distributed to students enrolled in a subject, describing the topic, syllabus,timetable, lecturer’s contact details, learning outcomes, assessment tasks and marking criteria, modes of delivery, and relevant institutional policies.


86

Three of we researchers are academics who represent very different roles within the University of Wollongong. Because of the diversity of our interests and our work in the University, we wished to gather commentary and data for many purposes. Christine Brown, Academic Services Division, has responsibility and interest in ongoing professional development of discipline based colleagues, and via this project seeks to assist academics expand their understanding of the curriculum and its relation to their own research. Emily Purser, Learning Development Unit, has responsibility and interest in the development of student academic literacy. Glennys O'Brien is Director of First Year Studies in the School of Chemistry and came to the project with an initial need to map the curricula in subjects which followed on from First year Chemistry, both within and without of the discipline. The fourth member of our team, Lorna Jarrett, has a background in science (physics) pedagogy and is currently enrolled in a PhD in science education. The Teaching and Learning institutional structure at this University includes a tier at faculty level called the Faculty Education Committee, responsible for overseeing teaching and learning within each faculty. The Academic Services Division has staff representatives on these faculty committees. The Education Committee of the Faculty of Science (FEC) was the original forum from which this cross-disciplinary investigation arose, made possible by the connection of these three academic researchers and others on that committee. Furthermore, discussions within that committee had uncovered various faculty needs for information regarding teaching management and support across the subjects and degree programs offered by the faculty. Thus the FEC and the particular representatives on it at the time, their roles, their individual experience and especially their engagement and sense of greater purpose, was the catalyst for the project and the reason that it has a broad perspective.

PROJECT AIMS Generally curriculum mapping projects reported in the literature cover degree programmes as a whole. In contrast, this project has a ground up approach, gathering data and commentary about the individual subjects offered by the Faculty of Science. Thus this project is not so much course curriculum mapping, rather it is a detailed survey of subjects offered by the schools in the faculty for multiple reasons designed to inform many players and accompanied by the evolution of a robust yet flexible methodology. The primary purpose is to ensure students’ adequate development of core and transformative discipline concepts and skills by identifying and tracking where these are introduced and how they are elaborated from first to third year subjects and beyond. This ground up approach is important from the point of view that the subjects are part of a variety of degree programs, both within and outside of the Faculty of Science. The gathered data allows examination of whole degree programmes or any suite of subjects offered by a school or department for consistency of approach in any aspect of the curriculum offered. There were several other purposes for the project: To engage staff in rich conversations about their subject and their teaching. To provide the schools and the faculty with data to assist development, review and quality

assurance at any level of management. To identify impediments to learning so as to facilitate and improve collaboration between faculty,

academic services and other projects (eg, resources for mathematics support) offering student support.

To identify, develop and share the mapping methodologies. The sequence of the investigations was determined by the fact that although there are three schools within the faculty, the researchers began with gathering data and commentary on the subjects offered by the School of Chemistry, because one of us was based in that school. Thus the specific objectives of this first round of the investigations were: Develop and test the methodology Establish the data set for Chemistry subjects Record details of the experience of the investigators to inform the latter parts of the project. Specific outcomes reported in this paper are focussed on the science discipline perspective, other findings will be reported elsewhere. We report on the methodology, curriculum outcomes and commentary in chemistry and we reflect on valuable findings of the emerging process and our collective experience.


87

METHODOLOGY This section describes (i) gathering of information and commentary, (ii) analysis of recorded interviews and discussions, and (iii) storage, analysis and use of subject data. This investigation has ethics approval from the UOW Human Research Ethics Committee.

GATHERING DATA The gathering of commentary and data about each subject was carried out by a variety of methods, chief of which was a recorded interview with each subject coordinator concurrent with assembling available materials for that subject. The form of the interviews resulting from our cross disciplinary deliberations was a short list of 7 areas of concern and related questions to generate focused discussion with the teaching academics. To conduct the interviews two or three of the researchers visited each academic in their own office; offering an opportunity for discussion and reflection that busy academics do not normally get in their daily practice. The engagement was structured so as to be easy, pleasant, interesting and fruitful for them as well as for each of us. We have chosen an active interview stance (Holstein & Gubrium, 1997) in order to foster collegiality and to engage staff in deep conversations in both education and discipline terms. Sumison and Goodfellow (2004) caution against the use of superficial approaches, and highlight the need to develop sensitive techniques to allow for nuanced responses from staff. Further, the interview process with guide questions and concurrent gathering of data was established in order to minimise commitment of staff time. The issues and their guiding questions for the interviews related to: 1. Subject scope: core concepts, ‘transformative’ concepts, skills or competencies developed. 2. Learning outcomes: specific objectives or learning outcomes. 3. Assumed knowledge/ skills: essential knowledge or skills assumed, pre-requisites. 4. Maths concepts/ skills: key mathematical concepts needed. 5. Available learning and teaching resources: sharable resources. 6. Obstacles to student learning: key areas which are repeatedly difficult, main obstacles. 7. Relationship to the Graduate Qualities: particular graduate qualities developed? Data sources: Recorded interview with subject coordinator Recorded focus group discussion with students Subject outline Lab manual / subject handbook Other resource materials such as tutorial materials, support materials and assessments. Participants: Interviewers: usually two or three researchers, not just the faculty researcher alone. Interviewees: School of Chemistry academic staff as subject coordinators. Other lecturers: School of Chemistry academic staff providing content summary. School of Chemistry technical staff reporting on use of instrumentation in teaching laboratories. School of Chemistry lab technical staff summary of lab administration of subjects. Focus group Oct 2009 participants: Students from CHEM101/102, 2008, with P or C grade (P or

PASS = 50-64% total mark, C or CREDIT = 65-74% total mark). Information and data about subjects were also gathered informally. Academic staff members provided summaries of content they delivered in subjects they taught in but did not coordinate. Teaching laboratory staff, who are not often thought of as teachers but have a significant teaching role, helped researchers develop a summary of undergraduate instrumentation experience and laboratory management. It is important to recognise all involved in teaching and ensure a space for their input and perspective. To date student input on chemistry subjects has comprised of two focus group discussions with students a year after they had completed first year chemistry. Students were asked to volunteer for discussion in focus groups if they had achieved a pass grade (P, 50-64% final mark) or a credit grade (C, 65-74%) in CHEM102 the previous year. Credit or Pass students were sought as these students were considered able to reveal most how they struggled with current and past chemistry or chemistry dependent subjects. These students were sought from the subjects BIOL214 (The Biochemistry of


88

Energy and Metabolism), CHEM213 (Physical Chemistry) and CHEM214 (Analytical and Environmental Chemistry).The focus groups were held over a lunch hour in week 12 of Spring session 2009, with the students taking on pseudonyms and the discussion recorded.

ANALYSIS OF RECORDED INTERVIEWS AND FOCUS GROUP DISCUSSIONS The qualitative analysis method for the recorded staff interviews and recorded student discussion groups was informed by Boyatzis (1998), Gubrium and Holstein (2001), Mertens (2005), Pope, Ziebland and Mays(2000), and Rubin and Rubin (2005). Analysis was done by a researcher who did not participate in the majority of the interviews and who does not teach in the School of Chemistry. The purpose of this choice was to minimise the possibility of the researcher being unduly influenced by pre-existing ideas and assumptions. Interviews were audio recorded but not transcribed. This was because of time and budget constraints but also because the interviews had been designed to cover a number of areas of discussion and collect data that, while not applicable to the research reported here, would be required in other parts of research program; thereby avoiding the need to ask staff to participate in multiple interviews. Each interview was listened to at least three times in its entirety, and sections where discussion focussed on the topics under study were transcribed. On the first occasion, the interview was listened to with minimal pauses in order to gain a general impression of the topics of discussion. During this time brief notes were made of possible themes and significant sections, with times noted in minutes. This approach was informed by the work of Pope et al. (2000). About one week later each interview was listened to again and the order in which interviews were listened to was varied, again to minimise the influence of pre-existing perceptions. Before the second listening, the notes made during the first listening were read. A spreadsheet was used to make notes during the second listening. Analysis focussed on areas of discussion relating to details of subject content, skills taught, assumed knowledge, assessments, resources which could be shared and transformative concepts. However, other segments were included which were considered to be significant to the delivery of curriculum. These included: issues with which students struggled; perceptions of the subject by staff or students; and things that teaching staff found rewarding or frustrating. Each of these interview segments was transcribed and assigned one or more codes. Codes were allowed to emerge from the data: this initially produced a large number of codes which were organised into categories. After completing this process for all interviews, the spreadsheet notes were read and the interviews listened to again to ensure that segments under study were represented accurately and in sufficient detail. During this time the codes assigned to each segment were refined if necessary. A second simpler analysis was also conducted by the discipline researcher who listened to all recordings at least once and also focussed on the coding and time mapping from the first analysis to specifically identify issues with possible remedial action at first year. Limitations of the data analysis method include the possibility that the researchers may have failed to identify some relevant data, leading to bias in the findings.

RECORDING AND ANALYSIS OF SUBJECT DATA The original intention of the project was to store data within a content management system (CMS) which would be the repository for identified sharable resources and which could potentially allow some data analysis and mapping. This system was not to become available within a practical timeframe, so researchers began with spreadsheets to store the data and perform some simple data searching and analysis. Due to the fact that access to a CMS was anticipated, it was decided to store the data in spreadsheets rather than take the intermediate step of entering it into a database. Use of a database would also have made the data more difficult to access for staff not familiar with using a database. One spreadsheet was developed for chemistry subjects, within which several different worksheets were developed to cover different areas of details. The data gathered from each subject were entered into this spreadsheet. Worksheets were made for (1) subject outline data, (2) content synopsis, (3) lab manual experiment details (4) instrumentation usage. In each worksheet one row is used for the data of each subject under various headings in the columns. The worksheet for lab details gathered from


89

lab manuals provides multiple column entries for each experiment or lab activity as well as columns for more general data such as support materials available. In terms of deriving information from the spreadsheet, a Perl based single user web application to search the content of the spreadsheet for qualitative data analysis has been developed (Diment& Trout, 2009). No further specific tools for reporting from the spreadsheets have been developed to date. Although the spreadsheet is large, there are valid reasons for not moving on from this yet: Spreadsheets are easily accessed and read by virtually all who would be involved in any way. The total number of CHEM / NANO subjects covered is 19, this has not proven unmanageable. Document control is maintained by a lead person, those with data entry roles are limited and other

users read or use the data. Worksheets can be easily copied and rows and columns manipulated into another pattern to

answer new queries. Some data processing and reporting is possible via development of macros. The School of Biological Sciences will use the methodology, but are developing their own suite of

worksheets, some of which will be in common with those for Chemistry. This experience is in common with that of teaching support staff at Curtin University who have developed subject mapping and planning tools using spreadsheets (Oliver, 2010). As yet sharable materials reside still with coordinators within materials for individual subjects. In the long term these will become available to multiple subjects via the CMS which will support the learning management system. Links will be made from various subject eLearning sites to this repository.

OUTCOMES The methodology for data gathering, storage and analysis has proven reliable, worthwhile and strong. This suite of procedures is now being used for mapping BIOLXXX subjects, with some simple modifications to suit the requirements of that school. There is now a data set of information for chemistry subjects gathered into the one spreadsheet, accessible to all staff. Tracking of concepts, skills, various laboratory activities, assessment types from first to third year is now possible. The School has begun various deliberations using this data and associated commentary: Development of coherent management of assessment and related policies for laboratory classes

across CHEM2XX subjects. From both staff and student commentary, there is a very clear indication of problems associated

with lack of mathematics skills, in terms of the simplest of algebraic manipulations. Due to the value of student commentary from focus group discussions, it is intended to initiate a

program of such conversations with a broader group of students carried out once a semester continuing throughout their whole degree.

PROVIDING CLEAR CURRICULA Clear explicit mapping of concepts and skills in a subject and through a progression of subjects in a discipline helps students to link together concepts learnt in different settings. With this in mind, we formulated our main project aim which was to ensure adequate development of core and transformational skills and concepts. In order to achieve this, we have included gathering data specifically about concepts for each subject. Somewhat surprisingly at times this was not straightforward because, “as the coordinator of the subject you do not necessarily know everything that is being taught (in your subject) because you do not have a copy of the lecture notes ... What we don’t have is a one page summary of every single topic area.” Coordinators find they do not know the details of all content taught in subjects they are coordinating. Some coordinators had planned to gather such details, in fact this project is helping to fill the gaps. In fact analysis of commentary shows that academics do not have the opportunity or the time to look closely at what is happening in the subjects they do teach in or related subjects. Subject coordinators are unable to join the components of a subject together, even less so illustrate connections between subjects, allowing such connections to be made explicit to students. This situation points to an overall lack in the information provided to students about discipline specific subject clusters and whole degree programs, where concepts and skills have not been explicitly woven together.


90

Outcomes from this lack of connectivity for weaker students were highlighted by focus group commentary and exemplified by one student, Jane: “Sometimes when it (the concept or procedure or calculation) is taught or explained in a different manner, it does not really connect with the way (I have learnt it before)... and I just think what is this guy talking about. It is completely different from what I thought.” (With intoned agreement from another student, John.) While high distinction students will make the necessary connections themselves, struggling students need explicit details, commenting that it can indeed be confusing to have the same concept presented from a different angle. This is of concern, because presenting a concept from different points of view is generally perceived as a powerful learning tool. However, when differing approaches to a particular concept are not apparently compatible, because staff are unaware of the details of concept presentation and application in other subjects, there is a clear negative impact on students struggling coming to terms with this concept. From the chemistry subjects dataset we can now track development of concepts and skills. Derivatives of this mapping for specific subject clusters and/or degree courses can now be developed by subject or year or degree coordinators and made available to staff and students alike. In addition from staff commentary, we have identified the more general core and transformative concepts which can be regarded as the glue needed to bind subjects together. It is frequently the case that these are not made explicit and yet these could provide an ideal framework within which to cross reference detailed concepts and skills among subjects.

OUTCOMES FOR FIRST YEAR CHEMISTRY Of particular interest to one researcher is commentary indicating where developments in first year chemistry can be made to better assist student learning. Chemistry staff teaching in second and third year subjects comment that students struggle most due to their inability to recognise and apply first year content. As a result coverage in some second year subjects (notably Physical Chemistry) includes time spent reviewing first year material and consequently less new material is covered. Some staff have made first year chemistry resources available for revising (2nd year Analytical Chemistry) where, ideally, second year students would go over this material before session starts. However it is notable that weaker students tend not to use such resources till they perceive a need. Focus group discussion analysis revealed related concerns from the student perspective. It is often not clear to students why certain concepts and skills taught in first year chemistry are relevant to their later studies. Some students, especially those in applied degrees, were convinced first year chemistry finished at the end of their first year, and were influenced by comments from older students in the same degree program to the effect that “as you get more ...into your degree, you won’t use the chemistry knowledge.” Science faculty students do have a better perception that material learnt in CHEM101/102 will be used, but some reported reliance on memory rather than referring back to saved materials. As a result both groups of students appear to haphazardly manage their collection and storage of materials to assist future application of concepts and skills learnt at first year. In fact several students discarded what we would have perceived to be valuable materials. These same students, by the end of their second year, were of the mind that “teachers should enforce the fact that you will need and use this knowledge throughout your degree.” As a result of this commentary, further development of strategies in CHEM101/102 are planned to enhance content structures and systems already in place, encouraging students to cross reference subject content, concepts, examples, calculations, lab experiments. Provision for content cross referencing is already made via formatting of space within the subject handbooks, however to date little direction has been given about the use of these allotted blank spaces, and informal observation shows the use to be highly varied with some students not using the spaces at all. There is a need to provide structure to these spaces, and examples and advice to students in recording notes and adding cross referencing. Thus content already delivered via lectures in a clear and explicit framework will be more clearly linked to other learning activities and study and, ideally, the whole ordered package retained for reference in later years. Further, because of the detailed mapping available, explicit concept linking to second year chemistry subjects is now possible. Over the next few months, biochemistry subjects BIOL213/214, will also be mapped, this will allow explicit connections between CHEM101/102 and BIOL213/214 to be made.


91

CROSS DISCIPLINARY EXPERIENCE From our experience we can also write that although this research team may be viewed as diverse, in fact each speciality makes a valuable contribution to all aspects of the project and this diversity has been a strength rather than a weakness. Notwithstanding the contribution of each, there was a certain necessary momentum gained by the project when an academic staff member of the faculty and based in one particular school came on board as a major player. We would say it is essential to have such a member as part of the team. This helped open the way for approaches to be made to staff for the detailed interviews. This also provided the team with one researcher who had detailed discipline knowledge, detailed knowledge of the subjects and courses in the School, teaching and coordination experience over years 1 to 3, and sufficient standing in the school to gain the time commitment of research active busy academics. This researcher was also able and willing to commit to this research as their own research per se. We would advise that each school will need a leader as champion for the project and a keeper of the information base (spreadsheet), who collects up any extra new data each year after a subject has been delivered and maintains the information base as a living document.

CONCLUSIONS There is now a valuable data set of information for chemistry subjects taught at UOW derived from a very detailed analysis of gathered information from and commentary about the subjects. This data set is already proving useful in teaching management. There is also now an established and flexible methodology to develop such a dataset for subjects offered by the other disciplines within the faculty. From our experience mapping curricula at the subject level is proving to be a very worthwhile and interesting exercise as the richness of information available in the commentary and data becomes apparent. The definite sense of collegiality and sharing between the researchers and the interviewed staff, all of whom have intensive research programs, was also very encouraging.

REFERENCES Aabakken, L. & Bach-Gasmo, E., 2000. Data and metadata: development of a digital curriculum. Medical Teacher, 22(6), 572. Bath, D., Smith, C., Stein, S. & Swannet, R., 2004. Beyond mapping and embedding graduate attributes : bringing together

quality assurance and action learning to create a validated and living curriculum. Higher Education Research & Development, 23(3), 313-328.

Boyatzis, R. E., 1998. Transforming Qualitative Information: Thematic Analysis and Code Development. London: Sage Publicatiions.

Bruinsma, M. & Jansen, E., 2007. curriculum mapping: integrating multiple perspectives on the curriculum. Curriculum and Teaching, 22(1), 25-45.

Diment, K. & Trout, M. (2009) The Definitive Guide to Catalyst: Writing Extendable, Scalable and Maintainable Perl–Based Web Applications, 1st edition. Apress.

Gubrium, J. F. & Holstein, J. A., 2001. Handbook of Interview research: Context and Method. London: Sage Publicatiions. Harden, R., 2000. Curriculum mapping: a tool for transparent and authentic teaching and learning. Medical Researcher, 23(2),

123-137. Holstein, J. A. & Gubrium, J. F., 1997. Active Interviewing. in D. Silverman (Ed), Qualitative Research Theory, Method and

Practice (pp. 113 – 129). London: Sage Publicatiions. Hege, I., Siebeck, M. & Fischer M., 2007. An online learning objectives database to map a curriculum. Medical Education, 41,

1095. Mertens, D. M. (2005). Research and Evaluation in Education and Psychology: Integrating Diversity with Quantitative,

Qualitative, and Mixed Methods (2nd ed.). Thousand Oaks, CA: Sage Publications. Oliver, B., 2010. Curriculum mapping for course review. at http://dmai.cqu.edu.au/FCWViewer/view.do?page=11864 accessed

21-06-10. O'Neill, G., 2009. A programme wide approach to assessment: a reflection on some curriculum mapping tools. at

http://ocs.aishe.org/aishe/index.php/international/2009/paper/viewDownloadInterstitial/118/79 accessed 21 Jun 2010. Plaza, C.M., 2007. Curriculum Mapping in Program Assessment and Evaluation. American Journal of Pharmaceutical

Education, 71(2). Pope, C., Ziebland, S. and Mays, N. 2000. Qualitative research in health care: analysing qualitative data, British Medical

Journal 320, 114-116 Robley, W., Whittle, S. & Murdoch-Eaton, D. 2005. Mapping generic skills curricula: a recommended methodology. Journal of

Further and Higher Education 29(4), 321-330. Ross, N., 1999. AMEE - Guide No. 14: Outcome-based education: Part 4-Outcome-based learning and the electronic

curriculum at Birmingham Medical School. Medical Teacher, 21(1), 26. Rubin, H. J., & Rubin, I. (2005). Qualitative interviewing. Sage Publicatiions. Sumsion, J. & Goodfellow, J., 2004. Identifying generic skills through curriculum mapping: a critical evaluation. Higher

Education Research & Development, 23(3), 329-346. Wachtler, C. & Troein, M., 2003. A hidden curriculum: mapping cultural competency in a medical programme. Medical

Education, 37(10), 861-868. Willett, T. G., 2008. Current status of curriculum mapping in Canada and the UK. Medical Education, 42(8), 786-793.


92

DOES MARCEL MARCEAU HAVE A PLACE IN THE CHEMISTRY LABORATORY? Karma L. Pearce ([email protected]) School of Pharmacy and Medical Sciences, University of South Australia, Adelaide SA 5001, Australia KEYWORDS: chemistry, laboratory, peer led team learning, peer teaching, scientific literacy

ABSTRACT A flood resulted in the Chemistry laboratories being unavailable for the last 5 weeks of teaching. Alternative peer led student learning strategies were developed to deliver the learning outcomes. The first strategy required students to video the last practical they performed before the flood ‘Marcel Marceau’ style without reagents, but narrating what they did. The following week students interviewed each other about their videoed practical. Fifty one percent of students felt the videos clarified ‘new knowledge’, while 76% either agreed or strongly agreed that they were able to identify glassware, equipment and instrumental or ‘hands on’ technique. Eighty one percent of students indicated that they would have viewed the video before their practical class if they were available. The second strategy required students to teach the application of a scientific concept to lay adults. Seventy nine percent of students felt that they understood the concept better after explaining it to an adult. While nothing can replace the ‘hands on’ experience that students gain in the laboratory, an alternative student centred learning approach incorporating peer teaching through producing laboratory videos and teaching lay adults resulted in deep learning. Student feedback suggested these activities could be incorporated into future courses. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 92-97, ISBN Number 978-0-9808597-0-6

INTRODUCTION Considerable research into the qualitative and quantitative outcomes for higher education of peer assisted learning and peer teaching has emerged since the 1970s. The terms peer assisted learning and peer teaching are used inter changeably across the literature and are inconsistently defined. The terms are related to a variety of settings including teacher assisted schemes, peer mentoring and ‘near-peer’ and ‘co-peer’ teaching, where peer learners may be one or more students. An extensive literature review on peer teaching and peer learning research has been published by Topping (1996). This review however, draws almost exclusively on research into the implementation of ‘peer led team learning’ (PLTL) strategies. In PLTL, the student replaces the professional instructor in a class setting and peer teachers his or her co peers in selected aspects of the curriculum. The benefits of peer teaching and learning are typically categorised in the literature under the headings of pedagogic, socio-psychological and structural.

A PEDAGOGIC APPROACH TO PLTL Improved academic achievement of students in targeted curriculum areas is widely employed as a measure of success for PLTL as a pedagogic strategy in educational institutions (Goldschmid & Goldschmid, 1976; Topping, 1996, 2005; Whitman, 1988). A recent PLTL case study in tertiary chemistry programs has measured improvements in students’ academic performance in those courses and observed increased course retention rates where there has traditionally been a significant drop out rate (Lloyd, 2010; Stewart, Amar & Bruce, 2007; Tien, 2007). Students gain from peer teaching exercises because it requires much greater focus on the curriculum content. Students must plan to teach specific material and, therefore, must organise the content in such a way that it can be clearly communicated. This results in both increased cognitive repetition and deeper insight (Goldschmid & Goldschmid, 1976; Nichols, 1994; Topping, 2005; Whitman, 1988). To teach is to learn twice. The act of verbalising is more effective in reinforcing learning than merely preparing to teach (Whitman, 1988) and the obligation to teach, as opposed to merely presenting information to a passive audience, generates even greater cognitive demands and, ultimately, benefits for the peer teacher (Topping, 2005). Learners also achieve more through being taught through their peers (Cate & Durning, 2007; Whitman, 1988) This appears to be because of the cognitive proximity between them. Although difficult to quantify these benefits, it has been observed that learners come to a greater understanding and retention of subject matter because peer teachers are able to explain and discuss concepts at an

Pearce, Does Marcel Marceau Have a Place in the Chemistry Laboratory

93

appropriate level (Whitman, 1988). The social constructivist paradigm presents learning as an interaction between social and individual processes. Accordingly, PLTL increases cooperation and collaboration partly because peers are less threatening than expert instructors and also because they have a shared lexicon (Cate & Durning, 2007; Whitman, 1988) The corollary of this is less competition between individuals within the group and less isolation of individuals, resulting in greater sharing of ideas and discussion between students, which enhances understanding and stimulated the processes – all of which is translated by learners into usable knowledge ( Tien, Roth & Kampmeier, 2007; Whitman, 1988).

SOCIO-PSYCHOLOGICAL BENEFITS The socio-psychological benefits of PLTL were largely qualitative and measured through self-reporting by individual students or by observation of students by their supervising staff. The benefits for both peer teachers and learners where a PLTL approach or some permutations of PLTL have been implemented are variously reported across the wide range of disciplines. Benefits cited include, higher levels of understanding, improved motivation to study, greater self confidence and self esteem and enhanced communication skills. (Assinder, 1991; Cate & Durning, 2007; Depaz & Moni, 2008; Goldschmid & Goldschmid, 1976; Lloyd, 2010; Moni, Hryciw, Poronnik & Moni, 2007; Stewart et al., 2007; Tien et al., 2007; Topping 2005; Whitman, 1988). Developed or improved leadership skills have also been cited as a positive outcome (Assinder, 1991; Cate & Dunning, 2007; Depaz & Moni, 2008; Goldschmid & Goldschmid, 1976; Lloyd, 2010; Moni et al., 2007; Tien et al., 2007; Topping, 2005). Nicholls (1994) cites that issues experienced by expert instructors such as learner passivity or inattentiveness can still persist between learners and peer teachers. These issues impact on the experience of both the peer teacher and learner with regard to perceptions of competence, relatedness and autonomy (Cate & Dunning, 2007).

STRUCTURAL BENEFITS OF PLTL The use of PLTL is a strategy that not only potentially benefits students, but may also benefit the educational institution by alleviating teaching pressures, which have increased logistical challenges as class sizes expand without accompanying increases in staff or other resources Cate & Dunning, 2007; Stewart et al., 2007). A PLTL approach, however, is not without its own resource demands. The extent to which the expert teacher may be required to supervise and have input into training for peer teachers will vary depending on how PLTL is implemented and structured (Goldschmid & Goldschmid,1976).

PEER LED TEAM LEARNING Although most research reports positive outcomes from the use of PLTL, not all research reports an unqualified success. Persky and Pollack (2009) studied implementation of a peer teaching approach combined with a problem based learning approach to teach selected curriculum of a pharmacokinetic course to second year Pharmacy students. Students reported negative attitudes to the learning experience, and performance in the course was not observed to improve (Persky & Pollack, 2009). Sprat and Leung (2000) reported similar difficulties with a ‘Legal and Documentary English’ course which formed part of a final year program in translation and Chinese. Reduced levels of performance were observed in the students, while the students themselves reported feeling less motivated and confident. Sprat and Leung (2000) attributed the negative outcomes to study design and implementation issues PLTL approaches have been applied in a diverse range of educational contexts with an even broader variety of designs. Some studies have involved small teams of as few as three students (Depaz & Moni, 2008; Persky & Pollack, 2009) deliberately selected (and mixed) on the basis academic performance, while other studies have used much larger groups of 20 students or more, which represent groups of mixed abilities ( Lloyd, 2010; Stewart et al., 2007; Tien et al., 2007). Some studies recommended or included training for peer teachers (Tan Tee Hwa, 2009; Tang, Hernandez & Adams, 2004; Tien et al., 2007) , while other studies have tied the PLTL approach with the use of web technology (Ross & Cameron, 2007; Stewart et al., 2007; Tan Tee Hwa, 2009).


94

CONTEXT OF THE STUDY On September 21st, 2009, the Reid building which houses the chemistry laboratories for the Pharmacy and Medical Sciences programs at University of South Australia was flooded. Students had five remaining 3 hour practical sessions to complete as part of a 13 week round robin practical series for a Chemistry course prior to the flood. Given that the laboratory practical class rooms lacked power, water and drainage, other strategies needed to be implemented to achieve the learning outcomes. This paper will focus on two peer led strategies which were implemented to bridge this gap and replace the practical classes; Marcel Marceau (Salm, 2007) chemistry which involved miming and videoing the last practical performed before the flood and secondly conveying a key concept learnt during the course to an adult from a non scientific background which was also filmed.

METHODS Students enrolled in the second year food chemistry laboratory classes within the Nutrition and Food Science program were unable to complete their laboratory program due to a flood. Two key strategies to replace the practical sessions are outlined in this paper. The first of these strategies was ‘Marcel Marceau chemistry’ (named after the famous mime artist (Salm, 2007)). The students worked in pairs and were asked to ‘act out’ the last practical they performed in the rotation using the correct glassware and other equipment, but without reagents, gas or water. The students were required to narrate precisely what they had done detailing both the technique and reaction outcomes. This process was filmed by a staff member and put on the web for other students in the class to view. There was a three week delay between actually performing the last practical in the rotation and filming the ‘Marcel Marceau’ (Salm, 2007) version. The following week students were given the opportunity to view the videos and to ‘interview’ their classmates to seek clarification where required. Students were surveyed via a paper based survey during week 11 of the course to find out what they thought of the videos. The feedback was grouped and the analysed. Given that students were unable to produce the normal practical portfolio, students were examined through formative questions on all 13 practicals and two of the experiments were assessed in their final exam. Students were not given options in the final exam and hence were unable to avoid the laboratory based questions. The second strategy required the students to individually choose a concept from any part of the course, apply it to an ‘everyday situation’ and teach the concept and application to an adult from a non-science background. This activity was performed individually and videoed via their mobile phone or camera. Students were required to produce a video of 10 - 15 minutes length in which they could demonstrate both strategies as a teacher and evidence that the learner was gaining some understanding of the concept being taught. Lay adults were give a written questionnaire to provide an independent evaluation of their learning experience which was posted back to the lecturer in a reply paid envelope. The feedback was grouped and then analysed. As students varied in their choice of key concept, not all the concepts chosen for this activity were examined in the final exam. Assessment was based on how well the concept was applied to an every day setting and later explained, the level of understanding gained by the lay adult (both assessed from the video) and feedback from the adult via the questionnaire.

RESULTS Seven male and 21 female students studying Nutrition and Food Sciences were enrolled in a second year chemistry course. Ninety six percent of the students were less than 25 years old. Furthermore, 65% of students scored a credit or better for their first year chemistry courses. Given that students were unable to produce the normal practical portfolio, students were examined through formative questions on all 13 practicals. Two exam questions focused on the practicals; 82% of students achieved a minimum of 80%. In the previous year only 61% of students achieved a minimum of 80% for similar laboratory based questions with a slightly lower weighting (13% vs. 16% for 2009 vs. 2008 respectively; n=35). Each activity was surveyed. Students consistently reported that they valued the opportunity to construct the knowledge through the student centred activities. Student feedback on performing the ‘Marcel Marceau’ videos included, “This activity forced me to understand the techniques and chemical


95

reactions well enough to teach others”, “developed my team work skills”, “improved my communication skills” and “it made me feel great and really boosted my confidence as I did really think I wasn’t very good at chemistry”. Students cited drawbacks as “their voice was squeaky on the video” and “nerves”. Students reported that they gained significantly from watching other students on video complete the practical activities that they were not able to perform themselves. All students reported watching all of the ‘Marcel Marceau’ video sessions (including the video they performed themselves) with 36% reporting a decreased feeling of intimidation in learning from their peers. A further 51% felt the videos clarified ‘new knowledge’, while 76% either agreed or strongly agreed that they were able to identify glassware, equipment and instrumental or ‘hands on’ technique. Eighty one percent of students indicated that they would have viewed the videos before their practical classes if they were available. Surprisingly, 68% of students said they would have viewed a video produced by students even if a video of the practical produced by academics was available. The reasons behind this included, “The lecturers are too polite. They say ‘do this, don’t do that’...but the students will use ‘real’ language such as ‘if you do this you will ‘stuff’ the prac and won’t get results’. I tended to hear this!” Students also favourably reported increased levels of engagement with the practical when they were quizzed by their peers. They believed filming the practical three weeks after it had been preformed followed by an oral defence the subsequent week had facilitated better retention of knowledge and reflection. The second strategy, teaching a key chemical concept and explaining it to a lay adult, was also enjoyed by the students. Seventy nine percent of students felt that they understood the concept better after explaining it to an adult. They reported the challenge of ‘peeling back the layers’ of complex concepts in order to explain them to individuals without a grasp of chemistry forced them to achieve a deeper level of learning for themselves. Student feedback also suggested this activity could be incorporated into future courses as they found it a highly effective strategy to gain a deeper understanding of complex concepts. All the lay adults either ‘agreed’ or ‘strongly agreed’ that they gained a clear understanding of the concept as taught to them by the students. The adults commonly reported that they enjoyed the activity, with one saying that they would ‘never look at everyday cooking processes in the same way again’.

DISCUSSION In using an innovative PLTL curriculum design in which information was collaboratively constructed by the students by working together, appraising each other and gaining feedback from each other, students were able to gain the desired learning outcomes for the course despite 5 weeks of traditional practical classes not being available. This is evidenced by the majority of students (82%) achieving a minimum of a distinction for the 2 laboratory based questions in the course exam. Both questions required students to demonstrate a deep learning and assimilation of key concepts conveyed in the practicals accessed. (Similar laboratory based questions were completed in the previous year with a much small number of students achieving 80%). Although this could be due to the larger class size in 2008 (35 vs 28 in 2008 and 2009 respectively), the improvement in results in the 2009 cohort is most likely to be due to the increased interaction with the material by the students and the level of deep learning achieved. Analysis of student feedback of the ‘Marcel Marceau’ series of videos exercise indicated that students gained an understanding of the key concepts underpinning the practicals which they were unable to perform in the laboratory themselves. Furthermore, students gained skills in collaboration and team work, communication skills, critical enquiry and reflection. Boud et al. (1999) suggest that one of the reasons for this outcome was that PLTL strategies such as this provide students with an opportunity to plan and work together which required them to develop good collaborative skills with their fellow students. These strategies also provided increased opportunities for students to explore the context of the theoretical knowledge which underpinned the practical sessions. Students were then able to engage and reflect on the material independently when the lecturers are not directly involved in the creation of knowledge. Further, students gained a meaningful opportunity to communicate knowledge. Communicating knowledge does not normally occur to the same extent during conventional practical sessions. Additionally, the students’ articulation of their knowledge was critiqued by their peers, while


96

they had the opportunity to learn from adopting the reciprocal role (Boud et al., 1999). Increased course grades and improved attitudes and motivation to learning were consistent with research reported by Lloyd (2010) and Stewart et al. (2007). Persky and Pollack (2009) point out that PLTL approaches are often more time consuming than other learning approaches and members of the group do not always ‘do their part’. The Marcel Marceau activity was filmed during a timetabled ‘laboratory class’ with all students participating and the subsequent videos were viewed in students’ own time in preparation for a subsequent class. Students did not report any dissatisfaction with being asked to view the videos outside of class time. On the contrary, the discussion board on the course homepage was filled with favourable and encouraging comments with respect to the videos. Students generally reported a greater understanding of their chosen key chemical concept as a result of having to explain and teach it to a lay adult. As reported in the literature, students taking up the role of teacher required them to attain a much deeper understanding of the curriculum content. Students must be able to understand and then organise the content in such a way that it can be clearly communicated, and on this occasion, communicated not to a peer, but to an adult lacking underpinning knowledge of the topic (Goldschmid & Goldschmid, 1976; Lloyd, 2010; Nichols, 1994; Topping, 1996, 2005). Contemporary scientific literacy is dependant upon on capacity of communicators to understand and apply scientific concepts to everyday life (Burns, O’Connor & Stocklmayer, 2003). In this exercise, students experienced an even greater cognitive demand, through being required to apply their chosen concept to an everyday process, and then teach rather than simply present information (Topping, 2005; Whitman, 1988). The students had no prior warning that they would not be able to complete their practical course (due to the flood), the PLTL activities were implemented mid-course to replace the practical classes and the assessment activities for the course were changed from those outlined in the course information booklet. Where the students could have complained, they embraced these activities wholeheartedly. The students’ enthusiasm, under the circumstances, made it easier for the lecturer as there were 5 strategies in total to replace to practical program. As opposed to the findings of Boud et al. (1999); Nichols (1994) and Cate and Dunning (2007), the demand on the lecturer was higher as each individual ‘Marcel Marceau’ video had to be downloaded from the recorder and uploaded to the course home page (Boud et al., 1999; Cate & Dunning, 2007; Nichols, 1994). In addition, the videos produced on the students’ mobile phones and cameras were emailed to the lecturer in 9 different formats; IT support was required to download appropriate software to open these videos. A further downside to the ‘Marcel Marceau’ videos was that some students were self conscious, nervous or distracted by the sound of their own voice which they perceived as squeaky. However, this appeared to be a minor distraction as 68% of the students indicated they would view the students videos even if there was a video of the practical available produced by academics. Another potential limitation to the study is the nature of the student cohort. The majority of the students were under 25years old and were academically strong performers in Chemistry. There was no separate analysis of results for the 35% of students who scored less than a credit for their first year chemistry courses. Although the results of the current cohort were compared to those of the previous year, the exam questions in concurrent years were not identical and of different weighting. A more thorough analysis could be conducted with a control group within the same cohort of students. The study would also have to be completed with an older student cohort to gather data on the efficacy of these approaches with an older age group. In conclusion, as a consequence of the unforseen flood, PLTL approaches have been shown to deliver equally, if not better learning outcomes in terms of academic grades, and better outcomes in regard to student motivation and the development of team work and communication skills. Student feedback suggests that PLTL approaches could be incorporated into future courses.

REFERENCES Assinder, W. (1991). Peer Teaching, peer learning: one model. ELT Journal 45(3) 218-219.

http://eltj.oxfordjournals.org/cgi/content/abstract/45/3/218 Boud, D., Cohen. R. and Sampson, J. (1999). Peer Learning and Assessment Assessment and Evaluation in Higher

Education 24(4): 413-426.


97

Burns, T. W.,. O’Connor, D. J and Stocklmayer, S.M. (2003). Science communication: a contemporary definition. Public Understanding Science 12(2): 183-202. http://pus.sagepub.com/cgi/pdf_extract/12/2/183

Cate, O. T. and Durning S. (2007). Peer teaching in medical education: twelve reasons to move from theory to practice Medical Teacher 29(6): 591-599.

http://www.informaworld.com/smpp/content~db=all~content=a782874045 Depaz, I. and R. W. Moni (2008). Using Peer Teaching to Support Co-operative Learning in Undergraduate Pharmacology.

Bioscience Education ejournal 11: 1-12. http://www.bioscience.heacademy.ac.uk/journal/vol11/beej-11-8.pdf Goldschmid, B. and Goldschmid M. L. (1976). Peer teaching in Higher Education.Higher Education 5(1): 9-33. Lloyd, P. M. (2010). Strategies for Improving Retention of Community College Students in the Sciences. Science Educator

19(1): 33-41. Moni, R. W., Hryciw, D., Poronnik, P. and Moni, K.B. (2007). Using Explicit Teaching to Improve How Bioscience Students

Write to the Lay Public. Advances in Physiology Education 31 167-175. http://advan.physiology.org/cgi/content/abstract/31/2/167

Nichols, D. (1994). Issues in Designing Learning by Teaching Systems. AAI/AI-ED Technical Report No.107. Lancaster Lancaster University: 7.

http://74.125.155.132/scholar?q=cache:b1H4BPLjWgYJ:scholar.google.com/++Issues+in+Designing+Learning+by+Teaching+Systems+&hl=en&as_sdt=2000&as_vis=1

Persky, A. M. and Pollack, G. M. (2009). A Hybrid Jigsaw Approach to Teaching Renal Clearance Concepts. American Journal of Pharmaceutical Education 73(3).

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2703274/ Ross, M. R. and Cameron H. S. (2007). Peer assisted learning: a planning and implementation framework: AMEE Guide no.

30. Medical Teacher 29(6) 527-545. http://www.informaworld.com/smpp/content~db=all~content=a783692065 Salm, L. (2007). MARCEL MARCEAU (1923 - 2007) Encyclopedia of mime. Spratt, M. and Leung, B. (2000). Peer teaching and peer learning revisited. ELT Journal 54(3): 218-226.

http://eltj.oxfordjournals.org/cgi/content/abstract/54/3/218 Stewart, B. N., Amar, F. G. and Bruce M.R.M. (2007). Challenges and Rewards of Offering Peer Led Team Learning (PLTL) in

A Large General Chemistry Course. Australian Journal of Education in Chemistry 67: 31-36. http://74.125.155.132/scholar?q=cache:Y33ccnMnsDAJ:scholar.google.com/+Challenges+and+Rewards+of+Offering+Peer

+Led+Team+Learning+(PLTL)+in+A+Large+&hl=en&as_sdt=2000&as_vis=1 Tan Tee Hwa, D. (2009). Student Peer Teaching Strategy, Malaysia. In Search of Innovative ICT in Education Practices: Case

studies from the Asia-Pacific Region, UNESCO Bangkok: 1-25. Tang, T. S., Hernandez, E. J.and Adams, B.S. (2004). Learning and Teaching: A Peer-Teaching Model for Diversity Training in

Medical School. Teaching and Learning in Medicine 16(1): 60-62. http://www.ncbi.nlm.nih.gov/pubmed/14987176 Tien, L. T., Roth V., and Kampmeier, J.A. (2007). Implementation of a peer-led team learning instructional approach in an

undergraduate organic chemistry course. Journal of research in science teaching 39(7): 606-632. http://www3.interscience.wiley.com/journal/97518099/abstract?CRETRY=1&SRETRY=0 Topping, K. J. (1996). The effectiveness of peer tutoring in further and higher education: A typology and review of the literature.

Higher Education 32(3): 321-345. http://www.springerlink.com/content/n724l17269g79xkj/ Topping, K. J. (2005). Trends in Peer Learning. Educational Psychology 25(6): 631-645. http://www.informaworld.com/smpp/content~db=all~content=a727312969 Whitman, N. A. (1988). Peer Teaching: To teach is to learn twice Higher Education report No.4. Washington DC, Association

for the study of higher Education: 103


98

MEASUREMENT UNCERTAINTY AS A THRESHOLD CONCEPT IN PHYSICS Anna Wilsona, Gerlese Åkerlind a, Paul Francisa, Les Kirkupb, Jo McKenzie b, Darren Pearcec, Manjula D. Sharmad Presenting author: Anna Wilson ([email protected]) a Australian National University, Canberra ACT 0200, Australia b University of Technology Sydney, Sydney NSW 2007, Australia c Queensland University of Technology, Brisbane Qld 4000, Australia d School of Physics, The University of Sydney, Sydney NSW 2006, Australia KEYWORDS: uncertainty, threshold concepts, curriculum design

ABSTRACT We report on the initial findings of a study aimed at developing ways to address threshold concepts in the design of undergraduate curricula, involving academics in two disciplines (physics and law) from four Australian universitiesThe present paper compares two different processes by which physics academics identified and characterised a candidate threshold concept, measurement uncertainty, using student interviews and their own experiences as teachers. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 98-103, ISBN Number 978-0-9808597-0-6

INTRODUCTION It has been suggested that, within each discipline, there are a limited number of concepts that are ‘threshold’ in nature, so-called because they act as ‘conceptual gateways’ to disciplinary ways of thinking. Such concepts are proposed to form a subset of fundamental (or key) discipline concepts, distinguished by five criteria: that such concepts are (1) transformative, (2) integrative, (3) probably irreversible, (4) frequently boundary-defining and (5) potentially troublesome to learn (Meyer & Land, 2006). Because of their gateway nature, threshold concepts are considered to play a key role in the development of students’ disciplinary ways of thinking. Students who gain understanding of a threshold concept obtain “a transformed internal view of subject matter, subject landscape or even world view” (Meyer & Land, 2005 p.373), leading not only to new ways of understanding a subject area but a shift in the learner’s sense of identity. Students who fail to grasp a threshold concept find their learning path blocked, with no means to proceed. However, the very transformative and integrative nature of these concepts can make them troublesome to learn (Perkins, 2006). Incomplete understanding or misunderstanding of threshold concepts is likely to have long-lasting implications for students’ learning in the subject area, including their ability to apply their learning in new and unfamiliar contexts. Such incomplete learning creates a path-blocking effect and a subsequent push to rote learning (Davies, 2006). Such rote-learning may explain why students may be able to apply discipline methods well enough to pass an exam, but may not be able to adapt their learning to a new context or setting and may not acquire discipline ways of thinking. Teachers of physics (and other science disciplines) may recognise such instances in, for example, students who can apply the equations that express Newton’s law, but without letting go of a fundamentally Aristotelian view of the relationships between force and motion. Threshold concepts may thus provide a particularly valuable focus for curriculum design attention. However, the exploitation of such concepts as curriculum foci is not unproblematic. Although the idea of threshold concepts has proved to have widespread appeal to teachers in higher education, and Meyer and Land (2005) note that academics are quick to suggest threshold concepts in their own disciplines, it is not necessarily easy for academics to distinguish threshold concepts from key concepts. As yet, no clear strategies have been devised to assist teachers in making this distinction, nor in helping them identify what it is about a particular threshold concept that makes it troublesome for students. The development of such strategies is the focus of this paper.

Wilson, Åkerlind, Kirkup,McKenzie,Pearce,Sharma, Measurement Uncertainty as a Threshold Concept in Physics

99

Davies (2006) suggests that threshold concepts may be identified and characterised through comparison of the reflections of multiple experts, and comparison of the concepts of experts and students. Another approach is illustrated in two recent studies of threshold concepts in disciplines closely related to physics. Park and Light (2009) used semi-structured interviews to categorise types and levels of student understanding of atomic structure, aiming to better understand the possible threshold nature of that concept. In an analysis of interviews with undergraduate engineering students, Scheja and Pettersson (2010) found compelling evidence of the threshold nature of both ‘limit’ and ‘integral’ in mathematics. Such studies raise the potential value of student interviews to explicate threshold concepts. However, this approach is resource intensive, and so not to be undertaken lightly. This literature raises two potential ways to clarify the troublesome and path-blocking nature of threshold concepts – a resource intensive approach involving interviews with students, and a less resource intensive approach based on the prior experiences of teachers with student misconceptions in their own fields. This paper provides a comparison of the two approaches in one particular case, in which a group of experienced academics, all teaching in first year physics, were brought together to identify a threshold concept in physics and collaborate on targeted curriculum development.

SELECTING A THRESHOLD CONCEPT Initially, five physicists from four Australian universities, chosen to sample different university types and different geographical regions, were brought together in a one-day meeting facilitated by educational developers. The physicists were introduced to threshold concepts through the five criteria outlined above, which were illustrated through a concrete, everyday example (ethics). They were then asked to brainstorm key discipline concepts without reference to their possible threshold nature, and subsequently to identify candidates for threshold concept status using Meyer and Land’s criteria. They identified several concepts as fundamental to the discipline, listed in Table 1, where they have been loosely grouped into different categories. We note that more than one third of the concepts (all of those in the two categories related to modelling and observation) are not exclusively characteristic of physics – that is, they are not terms that can be associated primarily with the disciplinary language – and yet they were clearly perceived as crucial elements of the discipline. From this list, the academics selected the concept of uncertainty as the potential threshold concept to be used in this project. This was subsequently clarified to be measurement uncertainty, to exclude the concept of inherent uncertainty and incompatible observables in quantum mechanics. An analysis of measurement uncertainty in terms of Meyer and Land’s criteria suggests it is a suitable candidate. Table 1: Key concepts emerging from the physics brainstorm

Key concept Grouping Field, flux, force, momentum, entropy, impulse, energy, potential, temperature, induction, acceleration, wave-particle duality, conservation laws, space-time, gravity, relativity, equilibrium

Terms used to name key discipline concepts or concept clusters

First principles, diagrams, modelling, vectors, frames of reference, idealisation-reality

Terms related to modelling or the tools used in modelling

Significance, approximation, orders of magnitude, uncertainty, measurement Terms connected with the act of observation or measurement

1. Transformative There are several symptoms of students’ behaviour which suggest that a good understanding of the role of measurement uncertainty results in a transformation in a student’s thinking. Before a student has grasped the role of uncertainty in measurement, they see the outcome of an experiment as a single number (Buffler, Allie & Lubben, 2001). This means that comparisons are made between the values x1 and x2 rather than say x1 ± σ1 and x2± σ2 (where σ1 and σ2 are measures of uncertainty on x1 and x2 respectively). Uncertainty is seen as a mistake – something to be eliminated or remedied, or which indicates an experiment has been performed incorrectly. When graphing data, lines are drawn to connect points rather than to show a trend. Once the threshold has been crossed, students experience a radically revised view of many aspects of measurement, including factors contributing to experimental design, the limitations on experiments and inferences from data, and indeed the very nature of experimental results. Uncertainty is seen as an intrinsic part of the result of a measurement and as essential in assessing the quality of the outcomes of an


100

experiment; its target magnitude becomes something that is an important criterion in the design of an experiment; and extrapolations/interpretations are made taking uncertainties into account. 2. Integrative The concept of measurement uncertainty integrates a range of concepts and skills in a way that makes more meaning out of the whole. Concepts such as random and systematic error, calibration, repetition, hypothesis testing, significance, tolerances, populations and samples, experimental design, the limits of what can be discovered, interpolation and extrapolation, modelling, approximation and more are brought together in a complex cluster to form a key element of scientific method. 3. Irreversible Once a student has grasped the role of uncertainty, their views of the interdependency of theory, experiment and data are irreversibly changed. Their ways of reading data change so that, for example, they distinguish between scatter and pattern, and they recognise all data as contestable. 4. Bounded or boundary-making Of the five criteria, the idea that threshold concepts help to define the boundaries of the discipline to which they belong is often the most difficult to interpret, and indeed the physics academics in this project decided to discard it as a criterion. Measurement uncertainty certainly does not serve to demarcate the discipline boundaries of physics from other sciences, instead comprising a cluster of ideas, capabilities and concepts shared by statistics, the sciences and many social science disciplines. It could, however, be seen as one of the ‘boundary’ concepts of quantitative, empirical studies. 5. Troublesome There is no doubt that uncertainty is frequently a troublesome concept for students to grasp. The mathematical formalism is non-trivial; the idea of quantifying something that by definition you are unsure of and cannot directly measure is deeply challenging; and learning to ‘read’ data is something that takes practice. In addition, the challenge to the idea of a ‘true’ or ‘exact’ value is often at odds with the definite language of theory (and hence lectures and textbooks). The realisation that data (upon which theories depend) are inherently uncertain, and that the process of measurement is imperfect, leads students naturally and compellingly to question the basis of physical knowledge. This can be deeply unsettling for students who crave clarity and certainty. A key reason why the academics chose measurement uncertainty as the subject of study (rather than previously identified threshold concepts such as gravity or entropy) was the importance they felt it should be accorded in the development of disciplinary thinking. An understanding of measurement uncertainty – that is, an understanding of how to identify different sources of uncertainty, quantify their effects, take those effects into account in planning experiments, analysing data and making logical inferences from those data, and an appreciation of the consequences of uncertainty – is one of the core characteristics and capabilities of an effective scientist. This is particularly true in physics, where many experiments are aimed at making precise quantitative measurements, and many theoretical predictions are expressed as numbers. However, traditional physics curricula often relegate uncertainties to the realm of the lab alone, or perhaps augment lab experiences with one or two supporting lectures or tutorials. Unfortunately, these activities rarely seem to engage students’ deeper learning. Indeed, for many students, the process of identifying, quantifying and propagating uncertainties is a tedious and occasionally mystifying chore, and a distraction from the real business of getting a result. More explicit pedagogical interventions may be required to provide students with structured opportunities to acquire this threshold concept.

UNCOVERING STUDENT CONCEPTIONS OF UNCERTAINTY In line with Davies’s suggestion, the lecturers first tried to use their own experiences and expertise to describe common student (mis)understandings of uncertainty. Their expectations, emerging from discussions at the initial one-day meeting, are shown in Table 2. Table 2: Physics teachers’ expectations regarding student conceptions of uncertainty Stage 1 No conception of uncertainty; no thought of it in relation to experimental outcomes Stage 2 Uncertainty is seen as mistakes, errors Stage 3 Uncertainty is seen as a means of quantifying how wrong you are Stage 4 Uncertainty is understood as something that must be planned for Stage 5 Uncertainty is a comprehensible, modelable, quantifiable, communicable result


101

The next step in the project was to investigate students’ actual understandings of measurement uncertainty. Through a series of web-based discussions and video conferences, the team used the outcomes of their discussions at the initial meeting and existing literature on teaching measurement uncertainty, particularly the work of Andy Buffler (Buffler et al., 2001, 2008), to design two scenarios to present to students for discussion in interviews. Scenario A (Figure 1) used a physics context (measuring the earth’s gravitational field) but only minimal description of the experiment, while scenario B (Figure 2) used a daily-life context (mobile phone battery lifetimes) and more experimental detail.

Figure 1: Scenario A, Magnetic Field Measurements Semi-structured interviews were carried out with 24 first year students (6 randomly-selected from each institution). During the interviews, Scenario A was presented first at two institutions, while Scenario B was presented first at the other two. In both cases, students were initially asked to comment generally on the data. With Scenario A, students were asked to comment on whether one group’s measurements were “better’ than the others; with Scenario B, students were asked to say which brand they would recommend to a friend. They were also asked if there was a true value of battery life or B field. Finally, they were asked for examples of situations where uncertainty might matter. At all stages, the students were asked to explain their answers, and interviewers followed up on concepts and difficulties they raised.

You go to a Magnetic Observatory where scientists are making sensitive measurements of the Earth’s magnetic field and they wish to compare these measurements with theories about the composition of the Earth. You go into a laboratory where two groups of scientists (group A and group B) are each busy with their own experiment to measure the magnetic field in the laboratory on that day. The table and graph below show the data gathered by each group.

Trial Group A Magnetic field (µT)

Group B Magnetic field (µT)

1 55 60 2 52 50 3 57 62 4 55 48 5 58 62 6 52 50 7 54 55 8 60 62

average 55.4 56.1


102

Figure 2: Scenario B, Mobile phone battery lifetimes

PRELIMINARY RESULTS The interviews were analysed phenomenographically (Åkerlind, 2005), with the aim of identifying variation in the ways in which students understand the concept of measurement uncertainty. This involved four physics lecturers and two educational developers experienced in phenomenographic analysis comparing and contrasting each of the interview transcripts in a search for key similarities and differences. The similarities enabled identification of common aspects of students’ understanding of uncertainty, whilst the differences highlighted variation in what some students noticed about uncertainty that others were unaware of. This analysis suggests that there are three distinct aspects to students’ understandings of uncertainty – a pattern-recognition element that allows students to distinguish between trends, noise and potential anomalies; a formal, procedural understanding that allows them to quantify and combine different elements of uncertainty; and a “meaning” element that invests uncertainty with a communicable meaning that has implications beyond the given data. A sophisticated understanding of uncertainty involved the integration of all three aspects, whilst a less sophisticated understanding emphasised only one or two of these aspects. Examples of student comments that focus on the pattern aspect include “Group A’s data is more constant than Group B’s,” “it goes up and then down and then up again. That is the first thing I noticed.” “… it looks like a parabola … actually I take back the parabola thing. It looks a lot more like a sine wave.” Some students explicitly made a connection between scatter and error, e.g. “It seems like there is a lot of error because it is not close. There is a wide change here… if there was less error the data would be closer together …” The difficulty that many students had in quantifying uncertainty (the formal aspect) is evident from responses such as “This is the worst part about physics, it’s working out that absolute or relative error, yeah, what is the error?…It’s either half the smallest measurement that you’re using…. Or else you do all these weird, complicated equations…,” “…that is how you calculate uncertainty by doing the

Battery life is a big factor that customers take into account when deciding which mobile phone to purchase. A consumer group wishes to advise potential purchasers of competing Sony-Ericsson and Nokia phones about the battery life of phones available from each manufacturer. The consumer group devise a standard test in which each phone is initially charged fully. For each phone, the display brightness is set to 50% and backlight is illuminated for 10s. used A landline is called with each phone. The call continues until the battery drains, and the phone shuts itself off. The consumer group test the Sony Ericsson K610i and the Nokia N72. The time for each phone to shut off is shown in the table and graph below (each mobile phone is recharged between trials).

Trial Nokia time to shut off (minutes)

Sony-Ericsson

time to shut off

(minutes) 1 335 346 2 362 404 3 332 416 4 335 295 5 337 310 6 402 288 7 380 340 8 362 402

average 355.6 350.1


103

equation of maximum value over the number of trials,” and again, “Minus one something from something divided by something?” An awareness of the meaning of uncertainty was most often evident in the students’ suggestions for causes of scatter or for situations where uncertainty might matter, such as this response from a student doing biomedical sciences: “…if I go on to do medicine, and I give someone morphine, two milligrams of morphine, you can say make you nice and happy, and the syringe is 10 mills plus or minus half a mill, that half a mill is going to matter. Because if I give them too little, it mightn’t do anything to them; and if I give them too much, it might just kill them,” and in this more prosaic but highly practical suggestion, “They should have uncertainty in their bus timetables.” The interviews suggest that it is the integration of these three elements into a coherent, interacting whole that characterises the threshold aspect of coming to understand measurement uncertainty. Most students recognised differences in spread in the data, and most were willing and able to suggest experimental and/or environmental factors contributing to the scatter and to differences between data sets. Most were also reluctant to imagine a ‘true value,’ instead using the term as a kind of short hand for an accepted or published value that they might compare results to. However, very few students were able to quantify their sense of the scatter, or employed disciplinary language such as mean or standard deviation when describing the data. This lack of integration is somewhat different from the stages in student understanding proposed by the physics academics, suggesting that the less resource intensive way of identifying threshold concepts, while potentially powerful, may miss some of the ways in which threshold concepts present barriers to conceptual change and development of disciplinary ways of thinking. Thus although both processes for analysing threshold concepts were valuable, in this study, the student interviews provided additional insights to guide subsequent pedagogical interventions. The next stage of the project (not described here) uses the understanding of student conceptions that resulted from the interviews to design different curriculum interventions aimed at improving student learning of the threshold concept. A common ‘post-test’ was implemented in each institution to assess the effectiveness of the interventions. Further details of the interview analysis and the results of the pedagogical interventions will form the focus of future papers and will be available at http://www. thresholdvariation.edu.au from late 2010.

ACKNOWLEDGEMENTS Support for this publication has been provided by the Australian Learning and Teaching Council Ltd, an initiative of the Australian Government Department of Education, Employment and Workplace Relations. The views expressed in this publication do not necessarily reflect the views of the Australian Learning and Teaching Council.

REFERENCES Åkerlind, G.S. (2005) Variation and commonality in phenomenographic research methods, HERD 24, 321-334. Buffler, A., Allie, S. and Lubben, F. (2001) The development of first year physics students' ideas about measurement in terms of

point and set paradigms. Int. J. Sci. Ed. 23, 1137-1146 Buffler, A., Allie, S. and Lubben, F. (2008) Teaching measurement and uncertainty the GUM way. The Physics Teacher 46,

539-543 Davies, P. (2006) Threshold concepts: how can we recognize them? In J.H.F. Meyer and R. Land (eds.), Overcoming barriers

to student understanding: Threshold concepts and troublesome knowledge (pp 70-84). New York: Routledge Meyer, J.H.F. and Land, R. (2005) Threshold concepts and troublesome knowledge (2): Epistemological considerations and a

conceptual framework for teaching and learning. Higher Ed. 49, 373-388 Meyer, J.H.F. and Land, R. (2006) Overcoming barriers to student understanding: Threshold concepts and troublesome

knowledge. New York: Routledge Park, E.J., and Light, G. (2009) Identifying Atomic Structure as a Threshold Concept: Student mental models and

troublesomeness, Int. J. Sci. Ed, 31, 233-258 Perkins, D. Constructivism and troublesome knowledge (2006). In: Meyer, J. H. F. & Land, R. (eds). Overcoming Barriers to

Student Understanding: Threshold Concepts and Troublesome Knowledge. Abingdon: Routledge, 33–47. Scheja, M. and Pettersson, K. (2010) Transformation and contextualisation: conceptualising students’ conceptual

understandings of threshold concepts in calculus. Higher Ed. 59, 221-241


104

CAN CREATING PODCASTS BE A USEFUL ASSIGNMENT IN A LARGE UNDERGRADUATE CHEMISTRY CLASS? Emma Bartlea, Nancy Longneckerb, Mark Pegrumc Presenting author: Emma Bartle ([email protected]) a School of Biomedical, Biomolecular and Chemical Sciences, The University of Western Australia, Perth WA, 6009 b Faculty of Life and Physical Sciences, The University of Western Australia, Perth WA, 6009 cGraduate School of Education, The University of Western Australia, Perth WA, 6009

KEYWORDS: podcast, social media, new media, undergraduate chemistry, student engagement

ABSTRACT Creating a three-minute podcast about a fundamental chemistry concept was set as a minor assignment in a large university introductory chemistry class with an enrolment of 352. Students were divided into groups of three and assigned the topic of either acids & bases or oxidation & reduction. Students worked as teams to produce a podcast and load it onto the class’s WebCT site as an attachment in a discussion thread. Students were expected to listen to six podcasts produced by teams from their own laboratory class and evaluate the podcasts using an online quiz based on criteria from the marking rubric. Student comments on WebCT and the anonymous class survey questions indicate that students considered this assignment a positive experience. It was done with minimal need for technical tuition on the part of the unit coordinator or demonstrators. These preliminary results encourage the authors to recommend similar assignments in other large, introductory science classes as a means of developing graduate attributes while maintaining development of content knowledge. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 104-107, ISBN Number 978-0-9808597-0-6

INTRODUCTION Communication skills are recognised as an important graduate attribute irrespective of discipline (Crebert, Bates, Bell, Patrick & Cragnolini, 2004). Scientists’ ability to communicate effectively is vital to their employment prospects, to their contribution to society and to society’s reception of science (Australian Council of Deans of Science, 2001; Jasanoff, 1998; Wellcome Trust/ MORI, 2000). Group work and interpersonal skills are also important graduate attributes. It is essential that scientists are able to work in collaboration with a diverse range of people across multidisciplinary fields, both within their organisation and the wider community (Towns, 1996, Towns, Kreke & Fields, 2000). This project is a product of the ALTC grant, ‘New media to develop graduate attributes of science students’ (Rifkin, Longnecker, Leach & Davis, 2010). Aims of the grant include identifying and developing teaching strategies and resources suited to large classes in science, such as creation by students of ‘new media’ like podcasts. Many of these new media have emerged as part of the shift from web 1.0, the informational web, to web 2.0, the social web, which involves active production rather than simply passive reception of media. There is considerable potential to link new media with contemporary learner-centred pedagogical approaches, where students learn through active engagement with content and with peers (Pegrum, 2009a; 2009b). Furthermore, new media have increasing relevance professionally and engage university students in authentic tasks and work-integrated learning (Rifkin et al., 2010). Podcasting requires a digital voice recorder and software to upload the recordings onto a suitable web page. Thus the basic technology is cheap, easy to use and portable. Apart from organisation, the podcast assignment described in this paper required little input from the unit coordinator or demonstrators and thus could be considered an efficient use of limited teaching resources. CHEM1105 Introductory Chemistry is a unit at The University of Western Australia with an enrolment of 352 in 2010. It is designed for students with little or no background in chemistry who wish to gain an understanding of basic chemistry. Students in this unit are enrolled in courses across all faculties at UWA, although most are associated with the Faculties of Engineering, Computing and Mathematics, Life and Physical Sciences and Natural and Agricultural Sciences. Hence, in order to motivate and engage students enrolled in this unit it is essential that students can be exposed to the

Bartle, Longnecker, Pegrum, Can Creating Podcasts be a Useful Assignment in a Large Undergraduate Class?

105

multidisciplinary applications of chemistry, including the relevance of chemistry to their specific courses of study. The unit is based on the Western Australian Tertiary Entrance Examination Chemistry course, and the entire content of the Year 11 course is taught in the 13-week semester. The large amount of material covered means there is a rigid lecture schedule and it is fairly inflexible in terms of deviating from the content during class contact hours. This paper details the development and implementation of a group podcasting assignment in the CHEM1105 Introductory Chemistry class. This assignment gave the students an opportunity to explore a chemical concept further or to explore different applications of that concept relevant to their chosen course of study. It is possible that students who develop an explanation of a fundamental concept will have a better understanding of that concept.

IMPLEMENTATION The group podcasting project was run over four weeks starting after the mid-semester break. Students were briefed about the project in a lecture and provided with an assignment handout via WebCT. The assignment handout detailed the podcast requirements and directed students towards websites on how to make podcasts. The unit is taught across two campuses, the local Crawley campus (metropolitan Perth) and the Albany campus (400km south of Perth) and all students enrolled in the unit completed the assignment. Albany students were able to download the assignment handout from WebCT, and listen to the lecture briefing through Lectopia. An example podcast on the topic of ‘atoms and chemical bonds’ was created and placed on the unit WebCT site so that students could listen to it and get ideas. ‘Atoms and chemical bonds’ is the first topic taught in the unit. Students were placed into groups of three by the unit coordinator based on their assigned bench in the practical laboratory class. Groups of three were chosen to ensure that if one person didn’t carry their weight for the assignment there was still a team of two to work on it. Because students in any group were in the same lab class, this ensured they had shared timeslots to work together on the project in the weeks when lab sessions weren’t scheduled. When it came time to upload the podcasts onto the unit’s WebCT page, each group was assigned a group name. To give students the perception that the assignment was meant to be a bit of fun, and also to help preserve the anonymity of the students, group names were based on characters from a commercially available chemistry card game, ElementaursTM (e.g. Princess Neo). A new discussion thread, ‘Podcast related’, was set up on the unit WebCT page so that students could communicate with their peers whilst doing the assignment, report any technical problems, and discuss any issues related to the assignment. WebCT was also used to post regular announcements and reminders as well as to supply students with detailed instructions on all the project requirements.

SUBMISSION WebCT was used for podcast submission. Folders for each laboratory class were set up on the discussion board and each group was required to post their completed podcast as a .wav or .mp3 file in their group’s folder, using their group name as the file name. Both the Albany and Crawley students posted their podcasts on the WebCT class site. Because each group had to submit their podcast as an attachment to a discussion post, there was the option to include a message. Several of the students added messages like ‘Enjoy! ’. Other humorous messages, such as ‘no backing tracks were harmed in the making of this podcast’, indicate that students found the assignment enjoyable and motivating. The podcasts were placed on the class’s WebCT discussion board during the final week of semester and were available for students to listen to during study week. Students were required to listen to six podcasts (that is, their own and five others) from their laboratory class group and comment on them.


106

TOPIC CHOICES The topics of ‘acids & bases’ and ‘oxidation & reduction’ were chosen as they are two of the major topics covered as part of the unit content. Also, past examination performance and anecdotal evidence suggested that students find these topics difficult and have common misconceptions. For example, students often confuse the relationships between oxidising agents and reducing agents, and oxidation and reduction processes. In the area of acids and bases, students often have trouble understanding the concept of acid strength as a measure of dissociation rather than concentration. Half of the groups in each laboratory class were assigned the topic of ‘acids & bases’ and the other half were assigned ‘oxidation & reduction’. The groups were given creative licence with the actual content of their podcast. They were told there were no strict guidelines on what they covered in the podcast as long as it was related to their allocated topic. They could choose to take one aspect of the topic from the lectures and explain it, develop an analogy to explain it, or they could find an application from their course of study, discuss it and explain how it related to their topic.

ASSESSMENT PROCESS The project was worth 5% of the overall unit mark. The remainder of the unit assessment comprised a final exam worth 50%, six laboratory sessions contributing 20% and 10 WebCT-based online quizzes as a form of continuous assessment worth a total of 25%. Although it was not worth much compared to the other assessment tasks, 94% of students completed the podcast assessment and, indeed, put a lot of effort into it. Students were required to complete a teamwork assessment, evaluating individual contributors to group assignments, and to submit this to their laboratory demonstrator during their last practical session. Only a few groups reported that one of the team members did not contribute to the assignment. Students were asked to sign a digital publication authorisation form to allow the podcasts to be published on iTunesU. Most students agreed to this and submitted the form with their assignment, but a few did not agree to having their podcasts made public in this way. The podcasts were assessed using a marking rubric. The five marking criteria were: 1) how well the introduction set the scene; 2) clarity, accuracy and relevance of content; 3) whether the conclusion provided a clear summary of the main points; 4) the structure and flow of the podcast and 5) technical sound quality (volume and clarity). Bonus marks were also awarded for creativity. The podcasts were marked by the unit coordinator. The marking rubric was turned into a WebCT-based online quiz and students were required to use this to assess a total of six podcasts from their laboratory class. They were required to self-assess their own podcast against the marking criteria, and also five other podcasts from their laboratory group. This included a total of three ‘acids & bases’ podcasts and three ‘oxidation & reduction’ podcasts. Students had 10 days to complete the peer assessment quiz. The open dates for the quiz fell during study week. Although completing the peer assessment quiz was voluntary and did not contribute any marks/weighting to their final mark for the podcast, 91% of the students who submitted a podcast completed the quiz. The marks given by the unit coordinator will be compared to the marks given by students with a view to using peer assessment as the sole form of assessment in the future.

STUDENT FEEDBACK In addition to the peer assessment quiz, some students replied to various podcast posts with written comments. These were always positive. Examples include: “That was one heck of a podcast! I really hope [the coordinator] chooses yours as an example! Great Work!”; “Wow!! Loved it loved it loved it!!!”; and “Genuinely entertaining. I’ve had a listen a few times just for another laugh. Great job!”. In addition, students made posts about the podcast assignments in the other discussion board threads, encouraging their peers to listen to specific podcasts which they considered to be outstanding. For example, one student wrote: “I just stumbled upon the most awesome podcast! Check out Wed 2-5 Lab group, Alum4 AB, it will seriously be worth your 3 minutes!”. This post received a reply: “Yeah big respect to this one...takes an uppercut!! Lol”.

Bartle, Longnecker, Pegrum, Can Creating Podcasts be a Useful Assignment in a Large Undergraduate Class?

107

CONCLUSION The authors are currently compiling student comments, feedback from an anonymous survey, and examination results for questions related to the podcast topics. These results will be analysed, written up and submitted for publication. Preliminary observations of the exam results indicate an improved understanding and learning of the ‘acids and bases’ and ‘oxidation and reduction’ concepts with an increase in the average exam mark compared to previous years. Student feedback has been positive enough to recommend use of this type of podcast assignment in other large science classes. The assignment required minimal effort on the part of the unit coordinator and demonstrators and so was an efficient use of limited teaching resources to provide an engaging learning opportunity for students. The assignment appears to have motivated students to develop an explanation of some aspect of a fundamental topic and to share their insights with their peers. As an engaging, learner-centred task, it fitted well with contemporary pedagogical approaches. Assessment of the podcasts using criteria such as clarity of expression and relevance was meant to explicitly emphasise the importance of specific graduate attributes.

REFERENCES Australian Council of Deans of Science. (2001). What did you do with your science degree? A national study of employment

outcomes for science degree holders 1990-2000. Centre for the Study of Higher Education, University of Melbourne. Crebert, G., Bates, M., Bell, B., Patrick, C-j, and Cragnolini, V. (2004). Developing generic skills at university, during work

placement and in employment: graduates’ perceptions. Higher Education Research and Development, 23(2), 147-165. Jasanoff, S. (1998). Coming of age in science and technology studies, Science Communication, 20, 91-98. Pegrum, M. (2009a). Communicative networking & linguistic mashups on web 2.0. In M. Thomas (Ed.), Handbook of research

on web 2.0 and second language learning (pp.20-41). Hershey, PA: Information Science Reference. Pegrum, M. (2009b). From blogs to bombs: The future of digital technologies in education. Perth: UWA Publishing. Rifkin, W., N. Longnecker, J. Leach and L.S. Davis. (2010). Students publishing in new media: Eight hypotheses – a house of

cards? IJISME, 18(1), 43-54. Towns, M. H. (1998). How do I get my students to work together? Getting cooperative learning started. Journal of Chemical

Education, 75, 67-9. Towns, M. H., Kreke, K., & Fields, A., (2000). An action research project: student perspectives on small-group learning in

chemistry. Journal of Chemical Education, 77, 111-5. Wellcome Trust/MORI. (2000). The Role of Scientists in Public Debate. London: Wellcome Trust.


108

UNDERGRADUATE RESEARCH AND INQUIRY ACROSS A ZOOLOGY CURRICULUM: AN EVALUATION THROUGH THE LENS OF EXTERNAL PEER REVIEW Susan M. Jonesa, Paula Myattb Presenting author: [email protected] aSchool of Zoology, University of Tasmania, Private Bag 5, Hobart, Tas 7001, Australia bTeaching and Educational Development Institute, The University of Queensland, Brisbane Qld 4072 Australia

KEYWORDS: inquiry, research‐led teaching, undergraduate research

ABSTRACT It has been established that student learning is greatly enhanced as a result of student engagement through research and inquiry based learning. However little emphasis has been given to how relationships between teaching and research are built within faculties or departments. External review of programs is a key strategy for building shared ownership of teaching programs. This project brought together a head of school and an external peer reviewer to carry out an evaluation of undergraduate research and inquiry across a curriculum, and to examine the student benefits. The first stage of the project, reported here, examined the extent to which undergraduate students are exposed to research and inquiry experiences within a department of Zoology. The approach utilised a mixed methodology including surveys and qualitative interviews with teaching staff. The reviewer identified a broad diversity of undergraduate research opportunities for students from first to third year, and a scaffolded approach to developing the students as researchers. In designing these learning activities, the teaching academics aimed to capture authentic research experiences for their students. This review ‘closes a loop’ between teaching and research within a department through critical evaluation of a program of undergraduate research opportunities. Stage Two of this project will focus on the student voices. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, pages 108-112, ISBN Number 978-0-9808597-0-6

INTRODUCTION It has been established that student learning is greatly enhanced as a result of student engagement through research and inquiry based learning (Brew, 2006). There is, therefore, a continuing emphasis on increasing the ways in which we link research and teaching within higher education and in how students can be included within the community of scholars (Brew, 2006). Much of the extensive literature on research-led teaching focuses on the student benefits. Less emphasis has been given to how the relationship between teaching and research is built within faculties or departments. Zetter (2002, p. 12) considers that “the relationship between research and teaching has to be created, planned and structured in a systematic way by departments”. He suggests that a detailed audit or review of a teaching programme can be designed specifically to identify research-led and research-based elements within the curriculum, and to “identify how the student experience of teaching and learning is linked to research” (Zetter, 2002, p. 13). A department may carry out a self-review with such terms of reference; however an alternative approach is to commission a peer review. Peer review of teaching is often considered as a means of improving teaching at the individual level or of demonstrating teaching excellence. However peer review focused on a group of “teacher-scholars” is a mechanism through which a shared accountability for teaching and learning can be fostered (Quinlan, 1996). A key factor in the success of such initiatives is clear leadership from within the relevant department by an academic who is a respected scholar and teacher (Quinlan, 1996). Peer review is particularly powerful if the evaluation is carried out by an external reviewer (Malik, 1996) as this facilitates the benchmarking of local practice against that of other institutions. In particular, curriculum benchmarking allows a department: “to identify best practices in relation to curriculum design and assessment with a view to learning from others and improving one’s own approaches to curriculum in the discipline” (Enhancing Assessment in the Biological Sciences, 2005). This project brought together a head of school (= department)(Jones) and an expert peer reviewer (Myatt) to carry out an evaluation of undergraduate research and inquiry across a curriculum, and to

Jones, Myatt, Undergraduate Research and Enquiry Across a Zoology Curriculum

109

examine the benefits that students gain from such an approach. During the first phase of this project we examined the question – To what extent are undergraduate students exposed to research and inquiry experiences within the School of Zoology at the University of Tasmania, and what are the student benefits? Stage 2 of this project, which is yet to be implemented, will investigate the impacts of the cohesive set of undergraduate research experiences (UREs) embedded across the curriculum by examining the benefits reported by students. This work builds on similar research at The University of Queensland (Myatt, 2009), whilst providing an insight into the unique features of zoology education at the University of Tasmania.

A DEFINITION OF UNDERGRADUATE RESEARCH Within the literature, there is a wide diversity of views on what is or is not defined as undergraduate research. Much of the debate is centred on the nature of the student activity and about the “newness” of the knowledge discovered. Although there is no “correct definition” (Beckman & Hensel, 2009) for undergraduate research, it is essential to define the term in line with our own aims and with the goals of the institution (Spronken Smith, 2010). For the purposes of this study, we defined undergraduate research as: “An inquiry or investigation conducted by an undergraduate student or group of students that makes an original contribution to the discipline or to the individuals involved.” (adapted from Beckman & Hensel 2009) This definition excludes research interactions where students are inactive or “passive” (Healey, 2005) such as in research seminars or research journal discussions; however it does include active participation in research activities where the students are seeking the answers to a research questions – whether in the laboratory, the field or the library. Figure 1 (from Healey, 2005) illustrates the framework that we used to examine and explain the diversity of possible UREs. Importantly we also consider undergraduate research includes research activities where the students uncovered knowledge that was original (new) to them, although not necessarily new to the discipline. This is an important distinction, and one that we made purposefully. We felt strongly that the student’s gains are dependant on a combination of features - the scientific authenticity of the task, the student’s sense of ownership of the research project and the student’s independence in performing it. These features did not include the newness, or otherwise, of the knowledge discovered.

Figure 1: A Framework through which to examine the diversity of possible activities that can be defined as “undergraduate research” (from Healey, 2005)

METHODS The project utilised a mixed methodology approach, using qualitative interviews and surveys to collect data. The external peer reviewer (Myatt) was hosted by the School of Zoology, University of Tasmania, for one week during April 2010. She was given access to printed teaching materials, including unit outlines and examples of assessment tasks for all relevant units taught by the School, and relevant evidence of the School’s approach to curriculum development (see Edwards, Jones,

Research-tutoredCurriculum emphasiseslearning focused onstudents writing anddiscussing papers or essays

EMPHASIS ONRESEARCHPROCESSES

STUDENTS AS PARTICIPANTS

Research-basedCurriculum emphasisesstudents undertakinginquiry-based learning

Research-ledCurriculum is structuredaround teaching subjectcontent

Research-orientedCurriculum emphasisesteaching processesof knowledgeconstruction in the subject

EMPHASIS ONRESEARCHCONTENT

STUDENTS AS AUDIENCE


110

Wapstra & Richardson, 2007; Jones, 2008; Jones & Barmuta, 2003). Teaching staff were invited to complete a short survey and to attend an interview with Dr Myatt. All interviews but one were face-to-face, with the exception being conducted by telephone; interviews were recorded and transcribed. A preliminary report summarising these findings was discussed with the Head of School (Jones), who provided further background information and rationale for teaching approaches before final analysis by the peer reviewer.

RESULTS AND DISCUSSION The primary focus of this study was the identification and examination of active research opportunities engaging Zoology students - both inside and outside of the curriculum. The eleven activities identified and examined in this study (Table 1) represent a broad diversity of opportunities, some voluntary but most embedded as compulsory assessment tasks within Zoology units. These begin with tasks available in the first year of a standard zoology course of study, and move through opportunities available in second and third year and conclude with a voluntary opportunity that is available across all years of study. An overarching characteristic of the UREs examined was the aim of the teaching academic to capture an authentic experience for their students. The authenticity was either inherent in the activity or was crafted as part of the learning design to ensure authenticity and thereby increase the effectiveness of the learning environment. It should be noted that while the individual authentic design features found in the learning activities were not seen as “unique”, the reviewer found that an overall consistent approach to authenticity was a pervasive element in the undergraduate research activities identified. The levels of authenticity encountered included: scientific posters used as an assessment task, with presentation of the posters at a student conference-like event with an invited audience, food and informal discussions; a scientific report submitted as a research paper manuscript, styled to a discipline-specific journal, and including scaffolded activities on writing and using the journal’s Guide to Authors; research projects based in research laboratories and with existing research groups; and volunteer opportunities to participate in the research of others as an ‘apprentice’ scientist. In many of the tasks examined, student engagement was encouraged through empowering students to identify their own research questions within a broader research topic. The opportunity to engage students in this process not only assists in increasing their motivation, but also enhances their critical thinking skills as they determine possible research questions within the confines of the topic. In these situations students’ choices were usually checked to confirm the validity and appropriateness of their selections. Students were frequently asked to define research questions, articulate strategies for answering those questions, conduct experiments, analyse data or appropriate discussion pieces. These steps replicate the scientific process, and enable students to gain an in-depth (and very real) appreciation of scientific research (Clark, Romero-Calderon, Olson, Jaworski, Loppato & Banjeree, 2009; see also Jones & Barmuta, 2003). Teaching academics reported that this level of authenticity meant that students encountered the unpredictable nature of science research. Gathering authentic primary data, in particular, can offer unexpected challenges and lead to unexpected learning outcomes. However, it is important to note that in learning activities in which students accessed existing data sets, the data were ‘real’ (that is, not fabricated but generated through research), thus enabling a level of authenticity to be achieved. The importance of making such explicit links between learning activities and staff research is emphasized by Brew (2007: p. 66). A feature of the overall program is the change of focus from “Students as Audience” to “Students as Researchers” (sensu Healey, 2005) from first to third year. Freestone and Wood (2006) commented that failing to introduce students to research in their first year represents a ‘lost opportunity’ which may impede students’ progression to postgraduate research.

Jones, Myatt, Undergraduate Research and Enquiry Across a Zoology Curriculum

111

Table 1: Examples of active research opportunities engaging students from years one to three of a degree major in Zoology

Course/title of activity Target students Specific/unique attributes

a KPZ163 Ethics workshop task (in Ecology)

1st year students Students hear ‘research stories’ from postgraduate researchers, then work in groups to design a hypothetical study that meets ethical guidelines.

b KZA215 Information literacy and critical analysis task (in Tasmanian Fauna)

2nd year students Students are asked to view a conservation-message video clip and critically analyse the message focusing on the scientific ‘facts’.

c KZA212 Scientific manuscript task (in Functional Biology of Animals)

2nd year students Students hear an authentic research conference presentation, provided with an authentic data set and then design their own research questions, analyse data and produce a manuscript.

d KZA225 Biology & Society (Prior to 2010: Evolution, Ecology & Society)

2nd year students A unit that focuses on critical analysis of the literature to enable discussion of contentious issues and synthesis of points of view.

e KZA360 five-day field excursion (in Conservation Biology & Wildlife Management)

3rd year students Students experience a 5-day immersion in a research context to experience a very real research environment, with the positive and negative experiences that arise.

f KZA355 Group research project (in Freshwater Ecology)

3rd year students Students self-select into research topic groups, devise research questions, propose methodologies and carry out the research.

g KZA301 Group research project (in Behavioural Ecology)

3rd year students Students carry out group research projects, from design, through execution and analysis, to reporting orally at a mock conference.

h KZA350 Research poster task (in Reproduction & Endocrinology for Conservation)

3rd year students A capstone experience within a unit; students synthesise published research, and produce a scientific poster within an authentic science conference environment.

i KZA304 Zoology Research Project 3rd year students Students work in a research laboratory environment with a researcher on a defined project for a whole semester.

j Undergraduate Zoology Research Volunteers Program

1st, 2nd & 3rd year students

Students choose from a range of authentic opportunities and work in a non-threatening environment with ‘real’ scientists.

k Summer Research Scholarships 2nd year students moving into 3rd year

A Faculty-wide program; students complete a small research project within a research group over 6-8 weeks and write a report.

However, in this program, there is an emphasis on scaffolding essential science skills and experiences, and a culture of placing ‘research’, in its many forms, in the student’s path. Teaching strategies similar to those reported here have been shown to create an appropriate context for learning, and to encourage a deep learning approach (Macfarlane, Markwell & Date-Huxtable, 2006); students gain high level, transferrable, learning skills through exposure to research-led teaching (Deakin, 2006). In conclusion, this review has ‘closed a loop’ between teaching and research by critically evaluating a program of undergraduate research opportunities embedded into the teaching of a university department through the lens of expert external peer review. Individual teaching academics benefited from active reflection on their teaching during interview, and the opportunity to seek advice or validation of their teaching approach from a critical friend. Thus, we have addressed the challenge of enhancing the link between departmental research and teaching through a structured review of teaching strategies designed to enhance that nexus (Zetter, 2002). Brew (2002) termed this “the backward glance”, meaning the act of teachers reflecting on knowledge gained in (research-led) teaching. The next stage of this project will focus on the student experience, and will seek congruences between the aims of staff in designing these learning activities, and the learning and developmental outcomes reported by their students.


112

ACKNOWLEDGMENTS This study was supported through the University of Tasmania Visiting Scholars Program. We thank the staff of the School of Zoology for their participation and contributions. The study was approved (H11157) by the Social Sciences Human Research Ethics Committee, University of Tasmania.

REFERENCES Beckman, M. and Hensel, N. (2009) Making Explicit the Implicit: Defining Undergraduate Research. Council on Undergraduate

Research (CUR) Quarterly, 29(4), 40-44. Brew, A. (2002). Enhancing the quality of learning through research-led teaching. Workshop presented at Annual Conference

of the Higher Education Research and Development Society of Australasia: Quality Conversations, Perth, WA. Brew, A. (2006). Research and teaching: beyond the divide. Houndsmill, Hampshire, Palgrave Macmillan. Clark, I. Romero-Calderon, R., Olson, J., Jaworski, L., Loppato, D., and Banjeree, U. (2009). “Deconstructing” scientific

research: a practical and scalable pedagogical tool to provide evidence-based science instruction. PloS Biology, 7(12), e1000264

Deakin, M. (2006). Research led teaching: a review of two initiatives in valuing the link between teaching and research. Journal of Education in the Built Environment, 1, 73-93. Retrieved June 16, 2010 from http://cebe.cf.ac.uk/jebe/volumes_index.php?edition=1.1

Edwards, A., Jones, S. M., Wapstra, E. and Richardson, A. M. M. (2007). Engaging students through authentic research experiences. Proceedings of the “Science Teaching and Learning Research including Threshold Concepts” Teaching and Learning Symposium. UniServe Science, Sydney 168-171.

Enhancing Assessment in the Biological Sciences (2005). http://www.bioassess.edu.au/ Freestone, R. and Wood, D. (2006). Exploring strategies for linking research and teaching. Journal for Education in the Built

Environment, 1(1), 94-111. Retrieved June 17, 2010 from www.cebe.heacademy.ac.uk/jebe/pdf/RobertFreestone1(1).pdf Healey, M. (2005) Linking research and teaching exploring disciplinary spaces and the role of inquiry-based learning, in

Barnett, R. (ed.) Reshaping the university: new relationships between research, scholarship and teaching,.30–42. Maidenhead: McGraw-Hill/Open University Press.

Jones, S.M. (2008). Enhance, Extend, Encourage: an incremental program for creating a whole-of-School community of researchers. Proceedings of the 2008 HERDSA International Conference, Rotorua, New Zealand.

Jones, S. M. and Barmuta, L. (2003). Challenging students to think differently: a science unit focusing on generic skills. Proceedings of the HERDSA Annual Conference, Christchurch, New Zealand. http://surveys.canterbury.ac.nz/herdsa03/pdfsnon/N1096.pdf

Quinlan, K. (1996). Involving peers in the evaluation and improvement of teaching: a menu of strategies. Innovative Higher Education, 20 (4), 299-307. Doi: 10.1007/BF01185805

Malik, D. J. (1996). Peer review of teaching: external review of course content. Innovative Higher Education, 20 (4), 277-286. Doi: 10.1007/BF01185803

Macfarlane, G. R., Markwell, K. W. and Date-Huxtable, E. M. (2006). Modelling the research process as a deep learning strategy, Journal of Biological Education 41(1), 13-20.

Myatt, P. (2009). Student perceptions of the undergraduate research experience: what do they think they really gain and how much influence does it have? In: Motivating Science Undergraduates: Ideas and Interventions. Uniserve Science Conference, Sydney, Australia, (85-90). 30 September - 2 October, 2009. Retrieved December 18, 2009 from http://espace.library.uq.edu.au/view/UQ:185676

Spronken Smith, R. (2010). Undergraduate Research and Inquiry-based Learning: Is there a difference? Insights from Research in New Zealand. Council on Undergraduate Research (CUR) Quarterly. (in Press)

Zetter, R. (2002). Making the department link between research and teaching. Exchange, 3, 12-15. Retrieved June 16, 2010 from www.exchange.ac.uk/issue3.asp

Rifkin, Longnecker, Leach and Davis, Blogging Biology and Podcasting Physics

113

BLOGGING BIOLOGY & PODCASTING PHYSICS: AUTHENTIC LEARNING VIA STUDENT CREATION OF NEW MEDIA Will Rifkina, Nancy Longneckerb, Joan Leachc, Lloyd Davisd Presenting author: Will Rifkin ([email protected]) aScience Communication Program, University of New South Wales, Sydney NSW 2052, Australia bScience Communication Program, University of Western Australia, Perth WA 6009, Australia cScience Communication Program, University of Queensland , Brisbane QLD 4072 Australia dCentre for Science Communication, University of Otago, Dunedin 9016, New Zealand KEYWORDS: new media, authentic assessment, graduate attributes, web, video

ABSTRACT The ALTC-funded ‘New Media for Science’ project explores ways to engage science students in authentic learning - to develop both their content knowledge and graduate attributes - via science communication. Specifically, we are looking at ways for students to create multi-media publications for the web and assessing the effectiveness of such learning activities. This paper offers twenty examples of new media assignments, including blogs, wikis, podcasts and video creation. We describe assignments that students are already completing for university classes in Australia and New Zealand as well as listing similar assignments that may be employed by science lecturers. The attractions of these methods of teaching and learning are addressed along with reported and perceived difficulties in conducting and assessing new media projects. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 113-115, ISBN Number 978-0-9808597-0-6

A NEED FOR COMMUNICATION SKILLS Universities have long recognised communication skills as a vital element in a graduating student’s set of generic attributes (Australian Council of Deans of Science, 2001; DEST, 2002). Over the last decade, studies of employer needs and expectations have identified a perceived lack of communication skills amongst graduates, especially in science (Raison, 2006). Responsibility for developing communications skills - and often graduate attributes, in general - is increasingly falling to science communication academics. It can be argued that integration of science communication into the science curriculum can strengthen contextualisation of scientific knowledge, with some contending that such integration can also deepen understanding of content. At the same time, the ubiquity of online and mobile communication technologies suggests that today’s students have many new opportunities to practice and develop their communication skills. They can publish their words, images, and conversations on the web. The internet allows for cost-effective publication, which enables student publication to become a mass learning activity, one that can be integrated into coursework rather than reserved for extracurricular efforts (e.g., an online version of the traditional student newspaper). Many more students can now learn how to understand and cater for target audiences as well as discovering the advantages of employing video, audio, images, social networking, and hyperlinked text - collectively referred to here as ‘new media’ - to enhance their communication. In participating in the production of new media, students engage in authentic and collaborative learning; they develop graduate attributes such as written and oral skills, teamwork, ethics, and critical thinking; and they gain professionally valuable knowledge and experience in science communication. However, while today's students may be ‘web orientated’, they are not as web capable as popular belief suggests, according to findings of the ‘Net Generation’ project funded by the Australian Learning and Teaching Council (Kennedy et al., 2007). These results echo observations and experiences of project team members in recent years. There is an opportunity to expand students’ general familiarity with new media into new skills and attributes. The ALTC New Media for Science project (Rifkin, Longnecker, Leach, Davis & Orthia, 2009) engages science lecturers and students in the production and publication of innovative and informative ‘new media’, concentrating on wikis, podcasts, blogs and videos that convey scientific content.


114

INNOVATIONS IN STUDENT ASSIGNMENTS Over the past year, we have gained insight into the range of new media assignments being undertaken in science classes in Australia and New Zealand. While some lecturers simply publish their own lecture notes, a large class of first-year chemistry students created team podcasts (Bartle, Longnecker & Pegrum, 2010). Some examples from this range of activity are not assessed, being employed mainly to give students practice in digital media communication. Others involve a formal assessment procedure. The project team has collected examples of new media activities and assessments from their own teaching as well as from interviews of ‘early adopter’ academics identified at conferences and through professional networks. Team members have also brainstormed a set of potentially worthwhile assignments, variations on existing teaching strategies. This set of new media assignments has been subdivided into four areas according to each one’s aim:

A. To practise skills in new media B. To focus on class content and teamwork C. To gain feedback from classmates or a target audience using new media D. To contextualise content – production of new media related to general science topics.

These activities encompass everything from simple PowerPoint presentations, to animations, the creation of audio and video podcasts, and the production of television or radio shows. Table 1: Existing & potential new media assignments for science students

A. Practising multi-media skills 1. Student creates a 1-minute podcast/video introducing themselves to the class 2. Student creates a short video of a person/place/item that is relevant to the subject 3. Student creates a 1-minute video on how to make a video 4. Student uses a blog to reflect and peer assess group work activities in the laboratory/field 5. Students critique classmates’ or lecturer’s videos using a ratings/comments system

B. Focus on class content & coordination of student activities (teamwork) 6. Student team creates a short (2–3min.) podcast to explain a scientific concept in the subject area or related to a class

field trip 7. Student team creates a lab report video, e.g. dissection or experiment 8. Student team creates an animation of a laboratory technique 9. Student team creates a wiki to explain a concept in science, e.g., element of the periodic table, or to produce a report,

class notes, or a literature review 10. Online collaboration with science students from other universities on wikis or Facebook sites

C. Using new media strategically, to attract attention/ gain feedback 11. Students create online PowerPoint presentations with opportunity for peer review 12. Students create a performance, e.g. science show for children on You Tube or a radio show on science topic for public

audience 13. Students create a webzine or a set of science news stories 14. Students create and promote an online event or product 15. Student creates a blog and promotes it online, monitoring the number of readers and comments

D. Contextualising content 16. Student interviews a science professional and uploads it to a website 17. Students create videos about young researchers, including research, storyboard, interview, editing, and publishing

online 18. Students blog regularly (3x per week) on a controversial science topic 19. Students review professional reports by scientists 20. Students (PhDs) produce a website promoting their research to industry.

CHALLENGES AND NEXT STEPS In gathering this range of new media assignments, we determined that such activities may be adapted across many areas of science and mathematics, be used as individual or team projects, and be assigned in large or small class sizes. Difficulties expressed by project team members and early adopters during the implementation of new media activities include: A lack of technical knowledge among students and lecturers on how to produce the media; that has

stimulated us to provide guidelines and examples, as well as outlining ways for lecturers to support students in finding their own way.

The need for suitable platforms to host student assignments; we are collating information about the major online platforms for new media publication, such as YouTube and FaceBook, to identify the best solution for each lecturer’s situation (different media and formats, large/small classes, need for peer feedback, …).

Rifkin, Longnecker, Leach and Davis, Blogging Biology and Podcasting Physics

115

Issues of privacy and copyright when publishing student work in the public domain; options for private publication range from totally-closed institutional platforms, such as Blackboard, to the major sites, such as YouTube or iTunesU, using their built-in groups and privacy settings.

New media activities are likely to differ significantly from other assessable tasks that lecturers assign in their science classes. Our research indicates that methods for providing valuable, constructive feedback and for assigning fair individual or team marks for new media assessments have been slow to develop. We recognise the need for building a ‘community of practice’: a network of academics using new media assignments in their science courses. Community members can share successes and pitfalls. To foster formation of the community of practice, the project team is assisting newcomers and early adopters over the coming year in adapting ideas to suit their individual classes, subject material, and desired outcomes as well as addressing problems and concerns. Guidelines are provided and concerns addressed in our New Media for Science wiki (Rifkin et al., 2010) (see http://newmediaforscience-research.wikispaces.com/ ). Our aim is to make basic activity guides for producing, viewing, sharing and assessing new media available at the SkillCity website (see http://skillcity.iaaf.uwa.edu.au), which provides opportunities for peer review. In sum, examples of how to use new media in science teaching, guidelines to address common concerns, and support for new adopters through a community of like-minded colleagues have been initiated in our project over the past year and are being built on and refined over the coming year. At the same time, we will be monitoring and measuring the impact on student learning offered by these approaches, to quantify and qualify the learning value that we have observed in our own classes over much of the last decade. If our hypotheses on the positive impact offered by learning of such assignments are shown to be true, we would like to see more science lecturers consider new media production as an element in their assignments. One could then expect improved development of graduate attributes, with a positive impact on student employability and ultimately enhancement of scientists’ communication skills and professional effectiveness.

REFERENCES Australian Council of Deans of Science, (2001). What did you do with your science degree? A national study of employment outcomes for

science degree holders 1990-2000. Centre for the Study of Higher Education, University of Melbourne. Bartle, E., Longnecker, N. & Pegrum, M. (2010). Can creating podcasts be a useful assignment in a large undergraduate chemistry class? In

prep. DEST, (2002). Employability skills for the future. Retrieved June 10, 2010 from

http://www.deewr.gov.au/Schooling/CareersandTransitions/EmployabilitySkills/Documents/EmpSkillsForTheFuture.pdf Kennedy, G., Dalgarno, B., Bennett, S., Judd, T., Gray, K.,& Chang, R. (2008) Immigrants and Natives: Investigating differences between

staff and students' use of technology. In Hello! Where are you in the landscape of educational technology? Proceedings ASCILITE Melbourne 2008. Retrieved June 10, 2010 from http://www.ascilite.org.au/conferences/melbourne08/procs/kennedy.pdf

Raison, M. (2006). Macquarie University: Science, Engineering and Technology Study. Macquarie University. Rifkin, W., Longnecker, N., Leach, J., Davis, L. & Orthia, L. (2009). Motivate students by having them publish in new media: An invitation to

science lecturers to share and test. 2009 UniServe Science Proceedings Retrieved June 10, 2010 from http://sydney.edu.au/science/uniserve_science/images/content/2009_papers/rifkin.pdf

Rifkin, W., Longnecker, N., Leach, J., Davis, L., Righetti, J. & Stewart, C. (2010). New Media for Science research wiki. Retrieved 10 June 2010 from http://newmediaforscience-research.wikispaces.com/


116

REFLECTIONS ON TEACHING COMPUTATIONAL PHYSICS AND APPLIED MATHEMATICS Paul Abbott ([email protected]) School of Physics, M013,The University of Western Australia, Crawley, WA 6009, Australia KEYWORDS: applied mathematics; computational physics; procedural programming; integrated computational environment; Mathematica

ABSTRACT In most applied mathematics and computational physics courses, simulation and modelling are taught by stressing numerical techniques, while visualisation often requires a range of specialised software tools. One approach is to use a procedural programming language such as Fortran or C. Although learning procedural programming is very useful it can detract from the desired goal of teaching computation. A second approach is to develop “black-box” applications for illustrating physical concepts. When well done this approach requires little instruction, and the focus is entirely on the situation under investigation. A disadvantage is that the student may not learn any computational techniques. A third way is to use an integrated computational environment, for example Mathematica, which couples an excellent graphical user interface to a high-level programming language. In this talk I will demonstrate my approach, developed over the last 20 years, by working through the solutions to selected problems from assignments and exams. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 116, ISBN Number 978-0-9808597-0-6


117

ALIUS: ACTIVE LEARNING IN UNIVERSITY SCIENCE - LEADING CHANGE IN AUSTRALIAN SCIENCE TEACHING Dan Bedgooda Mauro Mocerinob Mark Buntineb, Daniel Southamb, Marjan Zadnikc, Simon Pyked, Kieran Lime, Gayle Morrise, Brian Yatesf, Michael Gardinerf, Adam Bridgemang Presenting author Dan Bedgood ([email protected]) aSchool of Agricultural and Wine Science, Charles Sturt University, Wagga Wagga NSW 2678, Australia bDepartment of Applied Chemistry, Curtin University, Perth WA 6845, Australia cDepartment of Imaging and Applied Physics, Curtin University, Perth WA 6845, Australia dFaculty of Sciences and School of Chemistry & Physics, University of Adelaide, Adelaide SA 5005, Australia eDeakin University, Geelong/Melbourne Vic 3217, Australia fUniversity of Tasmania, Hobart Tas 7001, Australia gSchool of Chemistry, The University of Sydney, Sydney NSW 2006, Australia KEYWORDS: chemistry education, leadership, professional development

ABSTRACT The ALIUS project is about leading change in the teaching of chemistry in large university classes. ALIUS is a collaboration of six Australian universities funded by an ALTC Leadership for Excellence in Learning and Teaching Grant (LE8-818). The aims of this project lie in three domains: Development of project members as Science Learning Leaders Development of the skills of project members in practice-based learning and teaching innovation Creation of a virtual Learning Hub Methods being used to achieve these aims are: Leadership- professional development in leadership specifically targeted at fostering change in academic teaching practice Learning and Teaching Innovation - the U.S. NSF funded POGIL project will be used as a model for teaching innovation. Experienced POGIL instructors and facilitators brought from the U.S. run workshops and consultancies with ALIUS project members to

build member skills in teaching innovation. The learning hub (http://www.alius.edu.au/) will serve as a resource to share experiences in building teaching innovation, developed

materials, resources about innovative teaching methods This presentation will report results and experiences of the project, including: Experiences in implementing new teaching practices Experiences in motivating changes in teaching practice of colleagues Answers to uniquely Australian problems in implementation of POGIL style activities Report on impact nationwide Plans for next project Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 117, ISBN Number 978-0-9808597-0-6


118

PHYSICS PRACTICALS FOR ENGINEERING STUDENTS? Ragbir Bhathal ([email protected]) School of Engineering, University of Western Sydney, Sydney NSW 1797, Australia KEYWORDS: hands-on, simulations, physics practicals, physics for engineers

ABSTRACT Hands-on practical work in physics and engineering has a long and well established tradition in Australian universities. Recently, however, the question of whether hands-on physics practicals are useful for engineers and whether these could be replaced by computer simulations has been raised by some university administrators mainly due to cost cutting exercises. In order to ascertain the usefulness of first year practical physics classes to their engineering degree a retrospective survey was carried out with third year engineering students. This paper reports on the results of that survey. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 118, ISBN Number 978-0-9808597-0-6


119

DO OUR STUDENTS REALLY KNOW WHAT WE THINK THEY KNOW....AND WHAT CAN WE DO ABOUT IT? James Botten ([email protected]) School of Molecular & Biomedical Science, University of Adelaide, Adelaide SA 5005, Australia KEYWORDS: student perceptions, feedback, assessment criteria

ABSTRACT Despite the fact that academic staff spend many hours designing effective courses and information rich course manuals, it is often the case that some students will simply “miss the point”, that is they miss, or misunderstand, a vital piece of information that would allow them to excel in, and importantly, value and enjoy the course. Complications arise in the fact that which point is missed depends on the student in question. Thus changes made with the intent to improve the course for subsequent cohorts may result in a similar outcome i.e. some students will still miss the point! Despite this potential drawback, changes were made to aspects of the practical classes and related assessments to address the above problem in a 3rd year undergraduate course. Student evaluations were targeted at two specific aims; 1) to verify anecdotal observations that students value and enjoy practical classes, and 2) to identify specific concerns with the assessments, address the identified concerns and evaluate the effectiveness of the changes that were made. Encouragingly, the majority of students did value and enjoy the practical experience. Students’ concerns regarding the assessments were addressed and re-evaluated, revealing a mixed reaction, which will be discussed in further detail. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 119, ISBN Number 978-0-9808597-0-6


120

DEVELOPING SCIENTIFIC LITERACY FOR ALL UNIVERSITY GRADUATES Karen Burke da Silva ([email protected])

School of Biological Sciences, Flinders University, Adelaide 5001 SA, Australia

KEYWORDS: scientific literacy, graduate qualities, non-science students

ABSTRACT University educated students should possess at least a basic level of scientific literacy so as not to be disadvantaged in a highly technological global environment. Most, if not all Australian universities, list the graduate attribute of “being knowledgeable” yet many graduates are potentially unable to make decisions based on scientific understanding. Without a basic knowledge of science, graduates will find it difficult to make informed choices about their health care, their environment and the society in which they live. The ability to critically analyse the validity of a given argument or media presentation in order to come to a logical conclusion should be considered highly valuable in terms of graduate qualities and consequently an important requirement of all university degrees. A course to teach scientific literacy needs to be carefully structured to maintain student engagement and to provide understanding without the focus on creating scientists. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 120, ISBN Number 978-0-9808597-0-6


121

TEACHING AND FLEXIBLE LEARNING IN SCIENCE AND ENTREPRENEURSHIP COURSES USING MOBILE DEVICES Peter Cottrella, Rachel Wrighta, Lou Reinischb Presenting author: Peter Cottrell ([email protected]) aCollege of Science, University of Canterbury, Christchurch, New Zealand bJacksonville State University, Jacksonville, USA KEYWORDS: flexible learning, mobile devices, science entrepreneurship

ABSTRACT Mobile devices, such as iPods and cell phones, are a ubiquitous part of our world today. Young people have particularly embraced this technology and it is an integral part of their daily lives. The Science and Entrepreneurship courses, SCIE301 and SCIE302, offered at the University of Canterbury, use this technology to introduce students to how an idea in the laboratory can be developed in a commercial environment. These courses are designed for students, either on- or off-campus (distance) learners, from a broad range of disciplines. We have incorporated the use of mobile devices into SCIE301 and SCIE302 as teaching and flexible learning tools. The lecture content is provided as downloadable mp3 or mp4 files (video podcasts or vodcast format) for students to view in their own time. This course material is a combination of lectures, videos and PowerPoint presentations. The students are also provided with DVDs containing video case studies of individuals discussing their experiences in the world of entrepreneurship. The face-to-face lecture time is spent interacting with guest entrepreneurs and discussing the case studies. In this workshop we will present our innovative teaching methods that are facilitating flexible learning and discuss our future plans for utilising mobile devices in teaching and learning. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 121, ISBN Number 978-0-9808597-0-6


122

BUILDING A SUCCESSFUL OUTREACH PROGRAM Phil Dooley ([email protected]) School of Physics, University of Sydney, Sydney NSW 2006, Australia

KEYWORDS: science outreach, universities working with schools, recruiting science students ABSTRACT The School of Physics at the University of Sydney has an extensive and long-running outreach program. Here we describe three examples of best practice in terms of business model, content design and innovation. Three-hour syllabus-based workshops for year 12 students: “Kickstart” consists of hands-on workshops based around the NSW Higher School Certificate Physics syllabus. Nearly 2000 students per year participate, attracted by complex experiments from the syllabus (which actually work!) The University benefits as well as the students: 10 – 15% of first year physics students cite Kickstart as the major reason they chose Physics at the University of Sydney. An innovative meet and greet format: This format facilitates discussions between academics and participants by providing each academic with a poster with a question that participants can pose to them. The result has been startlingly successful for audiences of both high-school teachers and high-school students, with all participants engaging in unstructured and self-directed discussions with academics. Shows for lower high school students: The “Flying Freezing Floating Physics” show addresses the high attrition rate from science in the lower high school years, with a summary of the content of the senior Physics syllabus. Colour and excitement come from the use of charismatic experiments and multimedia, as well as career info and interviews. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page122, ISBN number 978-0-9808597-0-6


123

AN INTERCULTURAL EXPLORATION OF CONCEPTIONS IN THERMAL PHYSICS Helen Georgiou, Manjula D. Sharma Presenting author: Helen Georgiou ([email protected]) School of Physics, The University of Sydney, Sydney NSW 2006, Australia KEYWORDS: conceptions, cultures, thermodynamics, qualitative analysis

ABSTRACT An impressed enthusiast of thermodynamics once predicted that it would ‘never be overthrown’. Many years later, the beauty of thermodynamics has become increasingly elusive for students, hidden under a cloak of perceived obscurity and conceptual difficulty. This study aims to find what makes thermodynamics inherently difficult, or in fact if this is generally the case, by analysing a wide ranging sample of responses to a two part thermodynamics question. The analysis is an in-depth qualitative content analysis facilitated by the computer software NVivo. The sample includes responses from a range of ages from middle school to adult and from countries in Europe, North America and Africa, as well as Australia. A content analysis with a focus on the use of language in thermodynamics will be presented in this report, along with the implications for thermodynamics instruction. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 123, ISBN Number 978-0-9808597-0-6


124

TEACHING INSTRUMENTAL SCIENCE GLOBALLY USING A COLLABORATIVE ELECTRONIC LABORATORY NOTEBOOK Brynn Hibberta, Jeremy G. Freyb, Rosanne Quinnellc, Mauro Mocerinod, Matthew Todde, Piyapong Niamsupf, Adrian Plummera, Andrew Milstedb Presenting author: Rosanne Quinnell ([email protected]) aThe University of New South Wales, Sydney NSW 2052, Australia bThe University of Southampton, Southampton SO17 1BJ, UK cThe University of Sydney, Sydney NSW 2006, Australia dCurtin University, Bentley WA 6102 eThe University of Sydney, Sydney NSW 2006, Australia fChiang Mai University, Muang District, Chiang Mai, Thailand, 50200 KEYWORDS: electron laboratory notebook, science education, eResearch, eLearning

ABSTRACT In the higher education sector there is a strong push to improve the synergy between research and teaching. To achieve this there is a need to introduce into the undergraduate curriculum the new technologies that support research practice and process. There is no doubt that future scientific practice will increasingly involve collaborations around data and information that is delivered via the web. Our students must be trained in these new developments, and our staff must have access to tools that will facilitate their ability to teach it. New technologies, such as the Electronic Laboratory Notebook (ELN) developed at Southampton University in the UK, exploits the Web2.0 environment and offer the advantages of 1) being able to more readily share research resources, 2) as a digital record of experimental events and 3) a secure archive of data and metadata. We will discuss our initiative to extend the science curriculum in undergraduate chemistry through the introduction of an electronic laboratory notebook where instruments, experiments and data can be shared globally. The ELN is presently being implemented at UNSW, and the proposed project (funded by the Australian Learning and Teaching Council) will allow a multi-university (three in Australia, one in Thailand and one in the UK) exemplar of the ELN. By its nature, the project and its outcomes will be available worldwide for tertiary science training. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 124, ISBN Number 978-0-9808597-0-6


125

QUESTION GUIDED INSTRUCTION: A NEW TOOL TO IMPROVE THAI STUDENTS’ THINKING SKILLS IN THE PHYSICS EXPERIMENT CLASS Jiradawan Huntulaa, Ratchapak Chitareeb Presenting author: Jiradawan Huntula ([email protected]) aInstitute for Innovative Learning, Mahidol University, Bangkok, Thailand bDepartment of Physics, Faculty of Science, Mahidol University, Bangkok, Thailand

KEYWORDS: physics laboratory, students’ thinking skills, inquiry approach, guiding questions

ABSTRACT Inquiry based teaching has long been accepted as an effective teaching approach in science education. The inquiry approach has been shown to encourage students’ thinking skills in many disciplines. In this research, the inquiry approach is implemented through the use of guiding questions during instruction in physics experimental classes. The guiding questions are designed to challenge students and to encourage them to learn a physics experiment not only with their hands but also with their minds. They are embedded throughout the experiment, beginning to end. The guiding questions cover three particular aspects considered to be essential to succeed in learning while doing the experiment. The first aspect is relevant physics concepts, the second is the role of key equipment and the third is important techniques necessary to perform the experiment. The guiding questions are designed to encourage students’ thinking at the different levels corresponding to Bloom’s taxonomy. The study was conducted with 6 second year physics students from Thailand. The approach was evaluated using interviews and demonstrated that students’ thinking skills was better developed and they did engage with their minds in the physics experimental class. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 125, ISBN Number 978-0-9808597-0-6


126

WHY DO STUDENTS STILL BOTHER TO COME TO LECTURES WHEN EVERYTHING IS ONLINE? Jill Johnston, Dale Hancock, Vanessa Gysbers, Gareth Denyer Presenting author: Jill Johnston ([email protected]) School of Molecular Bioscience, The University of Sydney, Sydney NSW 2006, Australia KEYWORDS: iLecture, web-based lecture technologies, asynchronous learning, strategic learning styles

ABSTRACT With the recent emergence of an extensive range of online resources: everything from electronic lecture notes, slides, mp3 podcasts to the fully-downloadable recorded lecture with coordinated audio and visual images, the obvious question is: “Why do students still bother to come to lectures?” To explore this question, a preliminary survey was carried out within junior, intermediate and senior courses taught by School of Molecular Bioscience, The University of Sydney during second semester, 2009. This simple voluntary survey, which was mounted on WebCT, had two simple questions, each of which allowed both constrained and open responses. Do you attend lectures? How would you feel if there were no face-to-face lectures and lectures were only available online? Of those students who responded to the survey, the overwhelming majority, surprisingly, attended most lectures. For a voluntary on-line survey which was only accessible over a two-week period, the response rate was very encouraging. Most respondents also submitted abundant, enthusiastic free-form comments. The students were keen to give their opinion; many of the comments contained more than one reason for their attendance pattern. Because of this, the results (both numerical data and comments) provide a rich resource of student opinion for analysis. This conference provides the ideal opportunity to reflect on our data with assistance from a wider audience. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 126, ISBN Number 978-0-9808597-0-6


127

MAKING SCIENCE RELEVANT: A FACULTY-WIDE INITIATIVE TOWARDS ENHANCING THE STUDENT EXPERIENCE THROUGH AUTHENTIC LEARNING ACTIVITIES Susan M. Jonesa, Ruth Caspera, Julian Dermoudyb, Jon Osbornc, Brian Yatesd Presenting author: Susan M. Jones ([email protected]) aSchool of Zoology, University of Tasmania, Sandy Bay Tas 7005, Australia bSchool of Computing and Information Systems, University of Tasmania, Sandy Bay Tas 7005, Australia cSchool of Geography and Environmental Studies, University of Tasmania, Sandy Bay Tas 7005, Australia dSchool of Chemistry, University of Tasmania, Sandy Bay Tas 7005, Australia KEYWORDS: authentic learning, student engagement

ABSTRACT Authentic learning describes an educational approach in which students are presented with problems of real-world relevance within an environment that mirrors professional practice (Herrington & Herrington, 2006). There is substantial evidence that authentic learning is a powerful tool with which to encourage deeper learning and improve learning outcomes (Newmann, Secada & Wehlage, 1995). Our project aims to improve student learning outcomes and engagement by embedding authentic learning practice into the curriculum across our large and diverse faculty. We are employing a hub and spoke model of project management: the project team forms the hub, with discipline participants as the spokes, firmly connected to the rim of school-based colleagues. We have devised checklists that allow evaluation of units or learning activities against the principles of authentic learning articulated by Herrington and Herrington (2006). To date, we have run two workshops with discipline participants, who have received peer feedback on teaching initiatives they will trial in semester 2, 2010. In addition, we have initiated a faculty-wide scan for examples of good practice in authentic learning. Outcomes of this project will include: broad dissemination of the concept of authentic learning; a web-based resource of exemplars of authentic learning activities; and increased engagement of our students. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept29th to Oct 1st, 2010, page 127, ISBN Number 978-0-9808597-0-6

REFERENCES Herrington, A., & Herrington, J. (Eds.). (2006). Authentic learning environments in higher education. Hershey, PA: Information Science Publishing. Newmann, F., Secada, W. & Wehlage, G. (1995). A guide to authentic instruction and assessment: vision, standards and scoring. Madison, Wisconsin: Wisconsin Center for Education Research.


128

THRESHOLD LEARNING OUTCOMES FOR SCIENCE GRADUATES: A PROGRESS REPORT ON THE LEARNING AND TEACHING ACADEMIC STANDARDS PROJECT Susan M. Jonesa, Brian Yatesb Presenting authors: Susan M Jones ([email protected]) and Brian Yates ([email protected]) aSchool of Zoology, University of Tasmania, Private Bag 5 Hobart, Tas 7001 bSchool of Chemistry, University of Tasmania, Private Bag 75 Hobart, Tas 7001

KEYWORDS: academic standards; learning outcomes, LTAS; graduate outcomes

ABSTRACT The Learning and Teaching Academic Standards Project (LTAS) has been established by the Australian Learning and Teaching Council (ALTC) to engage discipline communities in the development of academic standards. As a demonstration project, the LTAS will lay the foundations for demonstrated achievement of learning outcomes by graduates, and will provide institutes with tools with which to develop standards-related processes. Discipline Scholars are leading this process, working with their discipline communities to define core learning outcomes for program/majors. These will be threshold standards, expressed as the minimum learning outcomes that a graduate of any given discipline must have achieved. As the Discipline Scholars in Science, we are working with the Australian Council of Deans of Science and a Reference Group of stakeholders drawn from the academic community, employer representatives and professional bodies. We also seek to engage more broadly with our discipline community via fora such as this UniServe Science Annual Conference. In our presentation, we will outline the scope of the LTAS and its relevance to current issues in the tertiary education sector. We will provide a report on our progress to date, and present draft Threshold Learning Outcomes for information and discussion. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 128, ISBN Number 978-0-9808597-0-6


129

MAPPING THE KNOWLEDGE STRUCTURE OF PHYSICS Christine Lindstrøm, Manjula D. Sharma Presenting author: Christine Lindstrøm ([email protected]) School of Physics, The University of Sydney, Sydney NSW 2006, Australia

KEYWORDS: knowledge structure, university physics, physics tutorials, teaching aids

ABSTRACT Most of science education research focuses on how to best teach science, whereas research into understanding the structure of various disciplines is rarer. This paper presents the findings of a careful analysis of the physics knowledge covered in first year university physics at the University of Sydney. The knowledge is characterised in terms of Bloom’s revised taxonomy, Bernstein’s description of knowledge structures as either hierarchical or horizontal, and in terms of Legitimation Code Theory, which – among other things – classifies knowledge in terms of its context dependence. We will discuss how and why first year students do not yet see physics as an integrated field, like their teachers do, and why Mechanics is the most suitable topic to begin the teaching of physics with – according to the knowledge structure of physics. I will also outline why we, in the sciences, have a spiral curriculum in which we return to the same topics year after year, unlike in the humanities. Finally, we will discuss the educational implications of this research and briefly outline the development and success of an educational environment influenced by these ideas. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 129, ISBN Number 978-0-9808597-0-6


130

MULTIMEDIA-BASED LINK MAPS – A PRELIMINARY REPORT

Nigel C. H. Kuan, Manjula D. Sharma Presenting author: Nigel C. H. Kuan ([email protected]) School of Physics, The University of Sydney, Sydney NSW 2006, Australia

KEYWORDS: physics education, multimedia, concept maps

ABSTRACT Past research has shown that students with lower prior experience in a subject area benefit greatly from the use of scaffolds in their learning. Part of this arises from the view that novices often have of subjects such as physics - a vast expanse of complicated and disconnected concepts and theorems. To address this, in teaching first year physics students with little or no prior knowledge, a particular approach using ‘link maps’ has been implemented at the University of Sydney. Separately, the steady proliferation of multimedia into teaching practice has also seen research emerge on the effective use of technologies such as video presentations and computer programs in teaching physics at tertiary and upper-secondary level. With a solid research foundation for these fields, we are interested in the synthesis of these ideas into a unique teaching and learning tool. Our research aim is to develop the fundamental ideas and research basis of link maps into video and computer-based multimedia, and investigate the effects of these tools on students. We put forth the developed multimedia tools, and preliminary findings of how they influence students’ understandings of physics. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 130, ISBN Number 978-0-9808597-0-6


131

USING CREATIVE ASSESSMENT TASKS TO ENGAGE STUDENTS IN LEARNING AND CONCEPTUALISING THE CHALLENGING CONTENT OF BIOCHEMISTRY Simon Myers ([email protected]) School of Biomedical and Health Sciences and School of Medicine, Campbelltown Campus, University of Western Sydney, Penrith South DC. NSW 1797, Australia

KEYWORDS: biochemistry, engaging assessments, conceptualising content, reduced failure rates

ABSTRACT Over the decades the subject Biochemistry has always been feared by university students as a very difficult, challenging and somewhat dry subject. It is a core second year unit that has to be passed for progression in most biological science degrees. Traditionally it has high failure rates and is often over assessed attributing to difficult student learning and lack of engagement with subject matter. Over the last two semesters, I have used creative assessment tasks to engage the students with the subject matter. The first task, in metabolic biochemistry, involved the construction of a pamphlet that was to be used in a medical centre to explain the cellular metabolism of diabetes. This task was conducted in groups of eight students and had to target 3 levels of audience; a medically trained person, a young adult with no medical or scientific training and a child under 10 years. In the second task in introductory biochemistry students were required to work in groups to produce a poster describing the process of protein synthesis as a fairy tale (nucleus to a functionally secreted protein). In both units the students have enjoyed the tasks, engaged with the subject matter and failure rates have substantially declined. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 131, ISBN Number 978-0-9808597-0-6


132

EPISTEMOLOGICAL BELIEFS, PERSONAL CHARACTERISTICS, AND HEALTH-RELATED BEHAVIOURS OF STUDENTS IN HEALTH AND HUMAN SCIENCES Steve Provosta, Stephen Myersb, Airdre Grantc Presenting author: Steve Provost ([email protected]) aSchool of Health and Human Sciences, Southern Cross University, Coffs Harbour NSW 2450, Australia

bSchool of Health and Human Sciences, Southern Cross University, Lismore NSW 2480, Australia

cTeaching and Learning Centre, Southern Cross University, Lismore NSW 2480, Australia

KEYWORDS: epistemological beliefs, big-five personality, health sciences, graduate attributes

ABSTRACT Although our graduate attributes often suggest that a university education will have some impact upon values and behaviours relevant to a student’s chosen area of study, in most instances we have little information about our students’ beliefs and characteristics either when they commence study or when they graduate. This discrepancy between our educational goals and knowledge of our students’ behaviour is especially problematic in disciplines such as psychology and natural and complementary medicine where “lay” views prevalent in the student population are most likely to diverge from the values and attributes given priority by academic educators. We surveyed all students enrolled in courses taught within the School of Health and Human Sciences (Nursing, Psychology, Natural and Complementary Medicine, Sports and Exercise Science, and Occupational Therapy) during second semester, 2010. The survey instrument included items assessing epistemological beliefs, approaches to studying, personality characteristics, health-related behaviours, and importance of graduate attributes. The outcomes of this survey will be discussed in terms of their implications for the assessment of threshold concepts underpinning graduate attributes and the development of curriculum to support their development in our degree programs Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 132, ISBN Number 978-0-9808597-0-6


133

INTEGRATED SCIENCE: AN INQUIRY BASED INTERDISCIPLINARY SCIENCE LEARNING EXPERIENCE Pauline Ross ([email protected]) College of Health And Science, University of Western Sydney, Sydney NSW 1797, Australia KEYWORDS: inquiry based teaching, integrated science learning, online modules, student learning experiences

ABSTRACT We know that fertile ground exists within our tertiary institutions for Science curriculum renewal and the long-term challenges of creating a broader epistemological, policy, pedagological and interdisciplinary curriculum base for learning science. This Science curriculum must meet the needs of our young people who have grown up in a highly technologised environment; interacting, engaging and disengaging with greater speed and choice than ever before. Integrated Science is an innovative new introductory science unit aimed at breaking the barriers and connecting the concepts between the traditional Science disciplines. The content covers hot topics in Science, which are important for our future and life on earth. Such topics are interdisciplinary, spanning physics, chemistry, biology with the central role of mathematics being emphasised and embedded throughout. To facilitate authentic and meaningful learning, this unit blends problem-based and inquiry-based pedagogies. Integrated Science is also fully on-line in Blackboard and modularised so that students have choice and flexibility in their learning. Assessment is aligned with the pedgagogy such that students develop integrated skills in literacy, numeracy, scientific thinking and communicating and conceptual understanding rather than rote learning and regurgiting facts. Currently our narrow conceptualisation of learning science is believed to be fueling the “flight from Science”, but is this really a “flight” from “rigour mortis” (McWilliam et al. 2008). Perhaps the main need is to validate a broader range of engaging Science learning experiences.

1. How do we integrate a broader inquiry-based pedagogy into a traditional Science curriculum? 2. How do we integrate the learning of science to integrate disciplinary silos? 3. How do we use a broader base of assessment strategies to align with our learning outcomes in a interdisciplinary framework?

Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 133, ISBN Number 978-0-9808597-0-6

REFERENCES McWilliam, E., Poronnik, P & Taylor, P (2008). Redesigning science pedagogy: Reversing the flight from Science. Journal of Science Education and Technology 17(5), 226-235.


134

THRESHOLD CONCEPTS: CHALLENGING THE WAY WE THINK, TEACH AND LEARN IN BIOLOGY AND SCIENCE

Pauline M. Rossa, Charlotte E. Taylorb, Chris Hughesc, Noel Whitakerd, Louise Lutze-Mannd, Vicky Tzioumisb Presenting author ([email protected]) aCollege of Health and Science, University of Western Sydney, Sydney NSW 2751, Australia; bSchool of Biological Sciences, The University of Sydney, Sydney NSW 2006, Australia cFaculty of Medicine, University of New South Wales, Sydney NSW 2052, Australia dSchool of Biotechnology and Biomolecular Sciences, University of New South Wales, Sydney NSW 2052, Australia

KEYWORDS: threshold concepts, teaching, learning, biology, science

ABSTRACT Meyer and Land (2003, 2005) proposed the notion of ‘threshold concepts’, which are central to the mastery of a specific discipline due to their transformative, irreversible and integrative nature. Using the methodology of Davies and Mangan (2007) we interviewed novice students (58) and expert academic staff (11) from three Australian universities and conducted an international survey of academics (55) to identify differences in novice and expert conceptions. We matched these data with understandings from the ‘misconceptions’ literature to create the ‘biology thresholds matrix’. The matrix demonstrates that threshold concepts in biology are not necessarily the troublesome content, but rather the tacit understandings of the discipline (Taylor, 2006, 2008; Ross & Tronson, 2007, Ross, Taylor, Hughes, Kofod, Whitaker, Lutze-Mann & Tzioumis, 2010). These are often not explicitly taught (Perkins, 2006) yet underpin difficult content areas including: energy and energy transformation, variation, probability and randomness, proportionality and surface area to volume ratio, dynamic equilibrium, linking the subcellular (submicroscopic) with the macroscopic, temporal and spatial scales (Ross et al., 2010), and the formulation and testing of hypotheses (Taylor & Meyer, 2010). These threshold concepts are not hierarchical in nature, but form a web of epistemes which has commonalities with tacit understandings in other science disciplines.

Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 134, ISBN Number 978-0-9808597-0-6

REFERENCES Davies, P. & Mangan, J (2007). Threshold concepts and the integration of understanding in economics. Studies in Higher Education, 32(6),

711–726. Meyer, J.H.F. & Land, R. (2003). ‘Threshold concepts and troublesome knowledge (1): linkages to ways of thinking and practising within the

disciplines.’ In C. Rust (Ed.), Improving Student Learning – Theory and Practice ten years on (pp. 412–424). Oxford: OCSLD. Meyer, J.H.F. & Land, R. (2005). Threshold concepts and troublesome knowledge (2): Epistemological considerations and a conceptual

framework for teaching and learning. Higher Education 49, 373–388. Perkins, D. (2006). Constructivism and troublesome knowledge. In J.H.F. Meyer & R. Land (Eds), Overcoming Barriers to Student

Understanding: Threshold concepts and troublesome knowledge. (pp. 33–48). Abingdon: Routledge. Ross, P. M. & Tronson, D. (2007). Intervening to create conceptual change. UniServe Science Teaching and Learning Research

Proceedings (pp. 89–94). Sydney: UniServe Science. Ross, P.M, Taylor, C.E., Hughes, C., Kofod, M., Whitaker, N., Lutze-Mann, L. & Tzioumis, V. (2010). Threshold concepts: challenging the

culture of teaching and learning biology. In J.H.F Meyer, R. Land & C. Baillie (Eds.), Threshold Concepts: from theory to practice (pp. 165–178). Rotterdam: Sense Publishers.

Taylor, C.E. (2006). Threshold Concepts in Biology: do they fit the definition? In J.H.F Meyer & R. Land (Eds.), Overcoming Barriers to Student Understanding: Threshold Concepts and Troublesome Knowledge (pp. 87–99). London: Routledge.

Taylor, C.E. (2008). Threshold concepts, troublesome knowledge and ways of thinking and practicing - can we tell the difference in Biology? In R. Land, J.H.F. Meyer & J. Smith (Eds.), Threshold Concepts in the disciplines (pp. 185–197). Rotterdam: Sense

Taylor, C.E. & Meyer, J.H.F. (2010). The testable hypothesis as a threshold concept for Biology students. In J.H.F Meyer, R. Land & C. Baillie (Eds.), Threshold Concepts: from theory to practice (pp. 179–192).Rotterdam: Sense.


135

SUPPORTING STUDENT LEARNING AND RETENTION IN PHYSICS, CHEMISTRY, MATHEMATICS AND COMPUTING – AN EVALUATION OF CURTIN UNIVERSITY'S SCIENCE CLINICS PROGRAM Elisabeth Settelmaiera, Marjan Zadnikb Presenting author: Presenting author Marjan Zadnik ([email protected]) a School of Education, Curtin University, Perth WA 6845, Australia b Department of Imaging and Applied Physics, Curtin University, Perth WA 6845, Australia

KEYWORDS: learning support tutorials, evaluation, student retention, science-, mathematics-, IT students

ABSTRACT The clinics were originally designed for students of physics, chemistry, mathematics and computing to enhance student learning and retention, particularly in their first year, and to identify students at risk early. An evaluation of the clinics was instigated in 2009. A survey was designed around issues raised by observations, in informal conversations with tutors (usually senior students) and with students who attended the clinics. The survey was administered to students attending clinics in all four disciplines- 49 students had responded by the end of April 2010. In addition, tutors and clinics coordinators were formally interviewed. Overall the results of the evaluation are positive: whilst clinics do not identify struggling students - since these students rarely attend, student feedback indicates that clinics have significantly improved student learning for those experiencing difficulties and who might otherwise have dropped out in the past due to a perceived lack of support and success. The clinics’ efficacy is evidenced through students’ tendency to attend clinics more than once and through positive student feedback on both clinics and tutors. The process of careful selection of tutors – based on tutoring skills rather than content knowledge – was identified as a crucial ingredient of the clinics’ success. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 135,ISBN Number 978-0-9808597-0-6


136

APPLYING THE ASELL FRAMEWORK FOR IMPROVEMENT OF A FIRST YEAR PHYSICS LABORATORY PROGRAM Salim Siddiquia, Daniel Southamb, Mauro Mocerinob, Mark Buntineb, Jo Wardc, Marjan Zadnika

Presenting authors Marjan Zadnik ([email protected]) and Salim Siddiqui ([email protected]) aDepartment of Imaging and Applied Physics, Curtin University, Perth 6845 WA, Australia bDepartment of Chemistry, Curtin University, Perth WA 6845, Australia cSchool of Science, Curtin University, Perth WA 6845, Australia KEYWORDS: introductory physics laboratories, student feedback, ASELL

ABSTRACT Physics 115 is a first-year non-calculus based unit offered to a wide range of students from various disciplines. The unit is taken by about 350 students per year, who have little or no background in physics. One of the assessment components of the unit is laboratory work which involves taking measurements, calculating uncertainties, performing data analysis, interpreting results and submitting formal written reports for assessment. In order to better understand students’ views on their laboratory experience, an extensive survey program was initiated by the project team in Semester 2, 2009. The survey data was analysed to investigate the characteristics of each of the six experiments. The results from the student responses indicated that two of the six experiments, “Simple Pendulum” and “Radioactivity Measurements”, needed revision. In order to obtain further detailed feedback from peers (students and staff from other universities), the two experiments were presented at the ASELL* Workshop held at the University of Adelaide in April 2010. As a result of the feedback from the ASELL Workshop, the “Radioactivity Measurements” experiment was immediately revised and presented to students in May of 2010. At the conclusion of the experiment, students’ feedback was once again collected and analysed. We will present the process, and results of the pre- and post- evaluation of this modified experiment, and demonstrate the effectiveness and power of the ASELL framework. *ASELL (Advancing Science by Enhancing Learning in the Laboratory) Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 136, ISBN Number 978-0-9808597-0-6


137

AN INVESTIGATION INTO SCIENTIFIC LITERACY AMONGST UNIVERSITY STUDENTS Michael West, Manjula D. Sharma, Ian Johnston Presenting author Michael West ([email protected]) School of Physics, The University of Sydney, Sydney NSW 2006, Australia KEYWORDS: scientific literacy, radiation, scientific process, physics

ABSTRACT In modern societies, science is central to governance and democratic involvement. Without understanding science, leaders and citizens cannot effectively participate in debates on issues from nuclear power to climate change. Scientific literacy is the establishment and assessment of benchmarks in public understanding of science. The dominant model is due to Miller (1983) and has three dimensions: core scientific content like “radiation” and “DNA”; understanding of the scientific process; and consideration of the relationship between science and society. We have created an instrument based on existing surveys for public scientific literacy, which have not been applied in Australia. The instrument covers the three dimensions outlined above. It was administered to 273 first-year Physics students with differing secondary science backgrounds, and 28 Honours and postgraduate Physics students. Preliminary results indicate marked differences in sophistication, focus and perceptions among the groups in some areas, and consistency in other areas despite very different levels of Physics education. We will present these results and further qualitative analysis carried out with NVivo. Proceedings of the 16th UniServe Science Annual Conference, University of Sydney, Sept 29th to Oct 1st, 2010, page 137, ISBN Number 978-0-9808597-0-6

REFERENCES Miller, J.D. (1983). Scientific literacy: A conceptual and empirical review. Daedalus, 112, 29.

TH UNISERVE SCIENCE ANNUAL CONFERENCElaplab.ucsd.edu/articles/Holcombe_Pashler_2010.pdf ·...

Documents

Transcript of TH UNISERVE SCIENCE ANNUAL CONFERENCElaplab.ucsd.edu/articles/Holcombe_Pashler_2010.pdf ·...