Collecting and Managing Multiple - Web viewVideo Data Sources for Visual Semiotic Analysis of...

Contemporary Approaches to Research in Mathematics, Science, Health and Environmental Science 20121

Collecting and Managing MultipleVideo Data Sources for Visual

Semiotic Analysis of Science Students’ Reasoning with Digital Representations

Joseph Ferguson

Deakin University

[email protected] paper proposes that in order to conduct a visual semiotic analysis of student’s use of digital representations, in this case NetLogo, multiple sources of video data must be collected and managed through the use of particular video data management software, specifically Camtasia Studio© and StudioCode©.

Natural Selection as a Complex Systems PhenomenonThe focus of the research addressed in this paper is on exploring the use of digital

representations, in this case multi-agent based computer modelling software (NetLogo), by students to reason about natural selection as a complex systems phenomenon. In particular, investigating the specific affordances of this technology for students’ reasoning about complex systems in science.

Natural selection is a particularly difficult scientific concept for students to comprehensively understand (Anderson, Fisher, & Norman, 2002; Dickes & Sengupta, 2012; Ferrari & Chi, 1998; White, 1997; Wilensky & Reisman, 2006). Research suggests that this is primarily because students do not understand the notion of a complex system, which is fundamental to natural selection (Centola, McKenzie,& Wilensky, 2000; Centola, Wilensky, & McKenzie, 2000; Chi, 2005; Chi, Roscoe, Slotta, Roy, & Chase, 2012; Dodick & Orion, 2003). In particular, many students believe that all systems must be deterministic and centralised, whereas complex systems are non-deterministic and decentralised (Resnick, 1991; Wilensky, 1993). If students can differentiate between a non-deterministic/decentralised system and a deterministic/centralised system then they can more productively explore the notion of natural selection.

Students’ ReasoningThe reasoning that is conducted by the students when using NetLogo is

conceptualised by the author according to Peirce’s (1998b) notion of reasoning. Peirce (1998b, p. 11) understood reasoning as “the process by which we attain a belief which we regard as the result of previous knowledge.” He proposed that while

FergusonCollecting & Managing Multiple Video Data SourcesContemporary Approaches to Research in Mathematics, Science, Health and Environmental Science 20132

there are various forms of reasoning that it is the scientific method that enables the most productive reasoning (Peirce, 1992b). Peirce (1992c) identified three different forms of scientific reasoning: deduction, induction and abduction. It was abduction, or hypothesis generation, that Peirce saw as the most important because it was the most generative for new understanding. It is scientific reasoning, specifically abduction, conducted by students that is the focus of this research because it is through abductive reasoning, the author argues, that students generate their own comprehensive understandings of complex systems and thus of natural selection.

Visual Semiotic AnalysisPeirce’s understanding of reasoning developed within a broader exploration of

reality and of the human experience. This was Peirce’s (1998a) pragmatism, and within this he developed a specific approach to the structure and the function of representations - a triadic conceptualisation of the sign system (Peirce, 1998c), which along with De Saussure’s (1983) dyadic conceptualisation of the sign system, forms the basis of semiotics. Of particular importance for Peirce (1992a, 1992d) was the relationship between the different parts of the triad (the object, intepretant, representamen) and their relationship to the different ways people conceive of reality (the firstness, secondness and thirdness of reality).

Peirce (1992e, 1998b) suggested that representations and reasoning, which he also referred to as logic, were intimately connected. He argued that reasoning is a fundamentally representational process - reasoning and representations form a kind of chiasmus. As Jappy (2013d) argues, it is possible to apply Peirce’s logic as semiotic (the intimate connection between representations and reasoning) to explorations of representational events in different contexts. As Jappy (2013a, 2013b, 2013c) demonstrates, this type of semiotic analysis is particularly suited to investigating visual representations and determining the relationships between objects, interpretants and representamens and how these are constituted as reasoning. The specifics of a visual semiotic analysis using Peirce are not addressed in this paper, rather the emphasis is on the way in which the necessary data can be collected and managed.

NetLogo as a Digital RepresentationNetLogo (Wilensky, 1999) is a multi-agent based modeling software package

that is designed specifically to allow simulation of complex systems. The software allows users to create and modify the variables that determine the behaviour of the individual components at the micro level as well as the variables that determine the role played by the environment (Blikstein & Wilensky, 2004; Levy & Wilensky, 2010; Wilensky & Reisman, 2006). The results of running the simulations are displayed in various forms (e.g. graphs, figures), which again can be modified, and which display the outcomes at the macro-level (Sklar, 2007). NetLogo is a digital representation because it has all the components of a digital representation stipulated by Peirce and functions via a digital medium.

The software is low-threshold, high-ceiling and so while the software is easy to learn to use in order to perform basic functions, it can also be used to conduct complex investigations of the same concept (Sengupta & Wilensky, 2011). It is

Contemporary Approaches to Research in Mathematics, Science, Health and Environmental Science 20133FergusonCollecting & Managing Multiple Video Data Sources

possible to both modify existing models of complex systems and create entirely new models using the software (Blikstein & Wilensky, 2010). As such, NetLogo is ideally suited for use by students to reason, in a scientific or Peircean way, about complex systems in a variety of different areas of science, including evolution (Dickes & Sengupta, 2012; Wilensky & Reisman, 2006)

Collecting Data – Multiple Video Data SourcesHow can students’ use of NetLogo be captured in such a way that the rich data

needed for a visual semiotic analysis is acquired? The data of most interest is not only the activity that is taking place on the computer screens, but also the interactions between the students and NetLogo, the interactions between the students as they use NetLogo and relate back to NetLogo and the interactions between the students and the teachers (again in relation to the use of NetLogo). Peirce’s logic as semiotic plays out in each of these situations and all of these events involves multiple representations, each consisting of a complex array of interacting representamens, objects and interpretants. All of this representational activity is relevant for a visual semiotic analysis and so as much of this data as possible must be collected.

The activity that was taking place on the computer screens was captured through the use of Camtasia Studio© (TechSmith, 2012), which has been shown to be the most effective screen capture software (Blevins & Elton, 2009; Carlson, 2009; Charnigo, 2009; Schnall, Jankowski, & St. Anna, 2005). Camtasia Studio© enables the capturing, in real time, and in high quality video form, of what is taking place on the computer screen, including any audio (Imler & Eichelberger, 2011, pp. 446-451). The result of using Camtasia Studio© was video of what took place on the computer screens, but only while the software was activated.

Camtasia Studio© has most often been used to create videos for the purposes of instructing computer users in the use of particular software and Internet sites (Blevins & Elton, 2009; Carlson, 2009; Charnigo, 2009). However, Camtasia Studio© has been recognised by some researchers as an effective means of collecting rich data about the human-computer interaction; in other words, what takes place on the computer screen (Goodwin, 2005; Hargittai, 2004; Hider, 2005; Imler & Eichelberger, 2011; Makkonen, Siakas, & Shakespeare, 2011; Schnall et al., 2005). This includes the use of Camtasia Studio© to record students’ use of NetLogo (Blikstein & Wilensky, 2010). The author suggests that Camtasia Studio© is the most effective means by which to collect rich data about students’ use of NetLogo because it records all the action that takes place on the computer screens, including in great detail students’ interactions with specific representations and the unfolding of logic as semiotic. This data is essential for a visual semiotic analysis.

Camtasia Studio© was also linked to webcameras attached to the students’ computers in order to capture the interactions between the students and NetLogo. This means that not only was the students’ use of NetLogo captured (what was taking place on screen) but also the students’ interactions with NetLogo (e.g. their gesturing at the screen as they used the software) was also captured. Camtasia Studio© enabled the simultaneous capturing of what was taking place on screen as well as the students’ interactions with NetLogo.

Ferguson Collecting & Managing Multiple Video Data Sources

Contemporary Approaches to Research in Mathematics, Science, Health and Environmental Science 2013 4

In order to more comprehensively capture the students’ interactions with NetLogo, as well the interactions between the students and between the students and the teachers, two tripod-mounted cameras were also utilised. One was focused on following the teacher in the classroom and capturing their interactions with the students and the other camera was focused on the students and their interactions with NetLogo, in particular their interactions with each other in relation to NetLogo. Each tripod-mounted camera was also linked to a lapel microphone. The teacher wore one microphone and the other microphone was placed on the desk near the group of students that was the focus of the research. The webcameras attached to the students’ computers also had built in microphones.

Below is a diagram showing the different video sources (as well as their associated microphones for capturing audio). In addition, student artfefacts were collected (the students’ work books) in order to capture the representations that were produced by the students. Unlike the video and accompanying audio, these were static representations, but they can still be subjected to the same type of visual semiotic analysis (discussed below).

CTrip mounted camera C SWebcamera S

MicrophoneCamtasia Studio C

Student SC

Teacher S

Figure 1: The use of multiple video data sources (as well as auditory data sources) to capture student’ use of digital representations (NetLogo).

Managing Data – Integrating Multiple Video Data SourcesOnce the multiple video data sources have been collected, as with all forms of

data, the video data must be managed in order to make it amenable for visual semiotic analysis. The author suggests that this can be achieved through the use of Camtasia Studio© in conjunction with Studiocode©. Just as Imler and Eichelberger (2011, pp. 448-449) make the conclusion in their study of user research behaviour that Camtasia Studio© and Studiocode© make a good software match in terms of data collection



and coding, the author suggests that Camtasia Studio© and Studiocode© make a good match in terms of managing data.

Studiocode© was originally developed to code video of sporting events (Imler & Eichelberger, 2011, p. 49). Studiocode© enables the tagging of events that occur in videos (Imler & Eichelberger, 2011). The result of using Studiocode© is a timeline on which relevant events are tagged, which can then be further analysed. However, it has since been used in many different research areas, including educational research, mainly to code events taking place in the classroom. This includes research conducted at the International Centre for Classroom Research at The University of Melbourne (Xu, Tytler, Clarke, & Rodriguez, 2012) and both the EqualPrime (Tytler, Hubber, & Chittleborough, 2010) and CRISP projects (Aranda et al., 2012), of which this research is a part, based at Deakin University. However, the author suggests that Studiocode© is also very useful for the initial management of the data.

In this research it was discovered that the audio from the webcameras was of very poor quality. This was initially fixed by utilising the audio adjustment tool in Camtasia Studio©, which allows the user to suppress the background/white noise and enhance the audio of interest. However, even after this audio adjustment was completed it was still very difficult to hear the students’ and the teachers’ dialogue. This problem was solved by pairing the audio from the lapel microphones (both the microphone worn the by teachers and the microphone on the students’ desk) with the video footage from the webcameras. This was achieved by removing the audio from the webcameras and stacking just the video from the webcameras with the video (and accompanying audio) from the tripod-mounted cameras. Stacking is a process in Studiocode© whereby one can pair one video with another video(s). The videos are placed on the same timeline and so run together. This enhanced the value (as a data source for analysis) of the video from the webcameras because the high quality video was now matched with high quality audio.

This example demonstrates the power of using Camtasia Studio© in conjunction with Studiocode© to manage multiple sources of video data. The stacking of different combinations of the different video data sources (and their accompanying audio) means that multiple viewpoints of a single event (meaning logic as semiotic or the playing out of representations and reasoning) were possible. In addition, it was possible to view multiple events occurring simultaneously. Thus it can be seen that through the use of both Studiocode© and Camtasia Studio© that multiple video data sources, and the relevant audio, can be turned into a data set that is useful for visual semiotic analysis.

Analysing Data – The Possibilities of Carefully Managed Multiple Video Data Sources

It is the combined use of screen capture software and video performance tagging software, specifically Camtasia Studio© and Studiocode©, which is proposed as an effective means of overcoming the methodological difficulties of conducting a visual semiotic analysis of NetLogo, in particular the difficult task of exploring students’ reasoning about complex systems as they interact with digital representations. Visual semiotic analysis of static representations (e.g. drawings), and



more recently of dynamic representations (e.g. video), using Peirce is a developing area of interest (as shown by Jappy (2013c). Beyond the relatively straightforward typology of the icon, index and symbol, the logic as semiotic of Peirce has not been used to conduct comprehensive visual semiotic analysis. And if it has been done it has only involved static representations, not dynamic representations.

The author suggests that while Peircean visual semiotic analysis of dynamic representations, including video footage of students’ use of digital representations to reason, may be a difficult, and at times daunting, task, it is worth pursuing. One possible way forward may be through the collection and then careful management of multiple video data sources and the associated audio (in conjunction with student artefacts). The author suggests that this could be achieved through the integrated use of Camtasia Studio© and Studiocode©.

ReferencesAnderson, D. L., Fisher, K. M., & Norman, G. J. (2002). Development and evaluation

of the conceptual inventory of natural selection. Journal of Research in Science Teaching, 39(10), 952-978.

Aranda, G., Aubusson, P., Chittleborough, G., Ferguson, J., Hoban, G., Hubber, P., . .. Tytler, R. (2012). Enhancing the Quality of Science Learning Using a Representation-Intensive Pedagogy (National Ethics Application Form) (pp. 1-34).

Blevins, A., & Elton, C. W. (2009). An Evaluation of Three Tutorial-creating Software Programs: Camtasia, PowerPoint, and MediaSite. Journal of Electronic Resources in Medical Libraries, 6(1), 1-7.

Blikstein, P., & Wilensky, U. (2004). MaterialSim: An Agent-Based Simulation Toolkit for Learning Materials Science. Paper presented at the International Conference on Engineering Education, Gainesville, Florida.

Blikstein, P., & Wilensky, U. (2010). MaterialSim: A Constructionist Agent-Based Modeling Approach to Engineering Education. In M. J. Jacobson & P. Reimann (Eds.), Designs for Learning Environments of the Future: International Perspectives from the Learning Sciences (pp. 17-60). New York: Springer.

Carlson, K. (2009). Delivering Information to Students 24/7 with Camtasia.Information Technology and Libraries, 28(3), 154-156.

Centola, D., McKenzie, E., & Wilensky, U. (2000). Survival of the Groupiest: Facilitating Students' Understanding of the Multiple Levels of Fitness through Multi-Agent Modeling - The EACH Project. Interjournal, 377.

Centola, D., Wilensky, U., & McKenzie, E. (2000). A Hands-on Modeling Approach to Evolution: Learning about the Evolution of Cooperation and Altruism through Multi-Agent Modeling - The EACH Project. Paper presented at the Fourth International Conference of the Learning Sciences, Ann Arbor, Michigan.

Charnigo, L. (2009). Lights! Camera! Action! Producing Library Instruction Video Tutorials Using Camtasia Studio. Journal of Library & Information Services in Distance Learning, 3(1), 23-30.

Chi, M. T. H. (2005). Commonsense Conceptions of Emergent Processes: Why Some Misconceptions Are Robust. Journal of the Learning Sciences, 14(2), 161- 199.



Chi, M. T. H., Roscoe, R. D., Slotta, J. D., Roy, M., & Chase, C. C. (2012). Misconceived causal explanations for emergent processes. [Research Support, Non-U.S. Gov't]. Cogn Sci, 36(1), 1-61.

de Saussure, F. (1983). Part One, Chapter 1 - Nature of the Linguistic Sign (R. Harris, Trans.). In C. Bally & A. Sechehaye (Eds.), Course in General Linguistics (pp. 65-70). London: Duckworth.

Dickes, A., & Sengupta, P. (2012). Learning Natural Selection in 4th Grade with Multi-Agent-Based Computational Models. Research in Science Education, 1- 33.

Dodick, J., & Orion, N. (2003). Introducing Evolution to Non-Biology Majors via the Fossil Record: A Case Study from the Israeli High School System. The American Biology Teacher, 65(3), 185-190.

Ferrari, M., & Chi, M. T. H. (1998). The nature of naive explanations of natural selection. International Journal of Science Education, 20(10), 1231-1256.

Goodwin, S. (2005). Using Screen Capture Software for Web Site Usability and Redesign Buy-In. Library Hi Tech, 23(4), 610-621.

Hargittai, E. (2004). Classifying and Coding Online Actions. Social Science Computer Review, 22(2), 210-227.

Hider, P. (2005). Coding online information seeking. [Article]. The Australian Library Journal, 54(3), 257-273.

Imler, B., & Eichelberger, M. (2011). Using Screen Capture to Study User Research Behavior. Library Hi Tech, 29(3), 446-454.

Jappy, T. (2013a). Medium Matters Introduction to Peircean Visual Semiotics (pp.107-136). London; New York: Bloomsbury Academic.

Jappy, T. (2013b). Modes of Representation Introduction to Peircean Visual Semiotics (pp. 79-105). London; New York: Bloomsbury Academic.

Jappy, T. (2013c). The Mute Poem Introduction to Peircean Visual Semiotics (pp.137-166). London; New York: Bloomsbury Academic.

Jappy, T. (2013d). Peirce Introduction to Peircean Visual Semiotics (pp. 53-78).London; New York: Bloomsbury Academic.

Levy, S. T., & Wilensky, U. (2010). Mining Students' Actions for Understanding of Complex Systems: Students' Explorations of Gas Models in the Connected Chemistry Curriculum. Paper presented at the Annual Meeting of American Educational Research Association, Denver, Colorado.

Makkonen, P., Siakas, K., & Shakespeare, V. (2011). Teaching Knowledge Management by Combining Wikis and Scren Capture Videos. Library Hi Tech, 28(5), 360-366.

Peirce, C. S. (1992a). Chapter 1 - On a New List of Categories. In N. Houser & C. Kloesel (Eds.), The Essential Peirce - Selected Philosophical Writings - Volume 1 (1867 - 1893) (Vol. 1, pp. 1-10). Bloomington: Indiana University Press.

Peirce, C. S. (1992b). Chapter 7 - The Fixation of Belief. In N. Houser & C. Kloesel (Eds.), The Essential Peirce - Selected Philosophical Writings - Volume 1 (1867-1893) (Vol. 1, pp. 109-123). Bloomington: Indiana University Press.

Peirce, C. S. (1992c). Chapter 12 - Deduction, Induction and Hypothesis. In N. Houser & C. Kloesel (Eds.), The Essential Peirce - Selected Philosophical Writings - Volume 1 (1867 - 1893) (Vol. 1, pp. 186-199). Bloomington: Indiana University Press.



Peirce, C. S. (1992d). Chapter 19 - A Guess at the Riddle. In N. Houser & C. Kloesel (Eds.), The Essential Peirce - Selected Philosophical Writings - Volume 1 (1867 - 1893) (Vol. 1, pp. 245 - 279). Bloomington: Indiana University Press.

Peirce, C. S. (1992e). Questions Concerning Certain Faculties Claimed for Man. In N. Houser & C. Kloesel (Eds.), The Essential Peirce - Selected Philosophical Writings - Volume 1 (1867- 1893) (Vol. 1, pp. 11 - 27). London: Indiana University Press.

Peirce, C. S. (1998a). How to Make Our Ideas Clear. In N. Houser & C. Kloesel (Eds.), The Essential Peirce - Selected Philosophical Writings - Volume 1 (1867-1893) (Vol. 1, pp. 124-141). Bloomington: Indiana University Press.

Peirce, C. S. (1998b). Of Reasoning in General. In N. Houser, A. De Tienne, J. R. Eller, C. L. Clark, A. C. Lewis & D. B. Davis (Eds.), The Essential Peirce - Selected Philosophical Writings - Volume 2 (1893-1913) (Vol. 2, pp. 11-26). Bloomington: Indiana University Press.

Peirce, C. S. (1998c). What is a Sign? In N. Houser, A. De Tienne, J. R. Eller, C. L. Clark, A. C. Lewis & D. B. Davis (Eds.), The Essential Peirce - Selected Philosophical Writings - Volume 2 (1893-1913) (Vol. 2, pp. 4-10). Bloomington: Indiana University Press.

Resnick, M. (1991). Overcoming the Centralized Mindset: Toward an Understanding of Emergent Phenomena. In I. Harel & S. Paypert (Eds.), Constructionism (pp. 205-214). Norwood, NJ: Ablex Publishing Corp.

Schnall, J. G., Jankowski, T. A., & St. Anna, L. A. (2005). Using Camtasia Studio© to Enhance Web Instruction Pages and Tutorials. Journal of Hospital Librarianship, 5(1), 77-81.

Sengupta, P., & Wilensky, U. (2011). Lowering the Learning Threshold: Multi- Agent-Based Models and Learning Electricity. In M. S. Khine & I. M. Saleh (Eds.), Models and Modeling - Cognitive Tools for Scientific Inquiry (pp. 141- 171). Dordrecht; New York: Springer.

Sklar, E. (2007). Software Review: NetLogo, a Multi-Agent Simulation Environment.Artificial Life, 13(3), 303-311.

TechSmith. (2012). Camtasia Studio (Version 8). Michigan, United States of America: Techsmith.

Tytler, R., Hubber, P., & Chittleborough, G. (2010). A Cross-National Study of Teaching and Learning in Primary Science Classrooms. Paper presented at the Contemporary Approaches to Research in Mathematics, Science, Health and Environmental Education, Deakin University, Burwood, Melbourn, Australia.

White, P. A. (1997). Naive Ecology: Causal Judgements about a Simple Ecosytem.British Journal of Psychology, 88(22), 219-233.

Wilensky, U. (1993). Connected Mathematics - Building Concerete Relationships with Mathematical Knowledge. (Doctor of Philiosophy Doctoral dissertation), Massachussetts Institute of Technology, Massachussetts Institute of Technology.

Wilensky, U. (1999). NetLogo (computer software). Evanston, IL: Center for Connected Learning and Computer-Based Modeling, Northwestern University.

Wilensky, U., & Reisman, K. (2006). Thinking Like a Wolf, a Sheep, or a Firefly: Learning Biology Through Constructing and Testing Computational Theories—An Embodied Modeling Approach. Cognition and Instruction, 24(2), 171-209.



Xu, L., Tytler, R., Clarke, D., & Rodriguez, C. (2012). The Value of Multi-Theoretic Analyses: Representational and Distributed Cognition Perspectives on a Classroom Sequence about Matter.

Collecting and Managing Multiple - Web viewVideo Data Sources for Visual Semiotic Analysis of...

Documents

Transcript of Collecting and Managing Multiple - Web viewVideo Data Sources for Visual Semiotic Analysis of...