1

Transforming knowledge and learning �through technologies and modalities: New forms of assessment

Wilhelmina Van Rooy Macquarie University

Eveline Chan

The University of Sydney

Paper code: VAN081043 Paper presented at the Annual Conference of the Australian Association of Research in

Education, Brisbane, 2008.

2

Transforming knowledge and learning �through technologies and

modalities: New forms of assessment

Wilhelmina Van Rooy, Macquarie University Eveline Chan, The University of Sydney

Abstract Scientific knowledge and the ways in which it is represented and communicated in its rapidly growing sub-disciplines are highly dependent on visualisations of complex phenomena, very often derived from computer-generated models and combined with high-speed computational power in outputting new information. The impact of digital technologies on transforming knowledge and its representation in the New Life Sciences is one such example, which has had an impact on how Biology is taught in classrooms today. For example, in the NSW Stage 6 Biology syllabus, aspects of molecular biology, bioinformatics and biotechnology are evident in the topics that cover DNA structure and function, genetic variation, and reproductive technologies in the core unit, ‘The Blueprint of Life’. The ways in which teachers access this knowledge and communicate it in classrooms, as is demonstrated by the other papers in this symposium, have been transformed by the information and communication technologies that have played a major role in the emergence of this new meta-discipline we describe as the ‘New Life Sciences’. While teachers and what gets taught in classrooms have been observed to be relatively responsive to shifts in disciplinary knowledge and practices, formal assessment structures in schools are often lagging in this respect. In NSW, at least, the paper-based tests which dominate formal examinations in senior science are heavily reliant on written responses to static, print-based representations which are predominantly coded in verbal text. New ways of representing and communicating scientific concepts in classroom practice necessitates new forms of assessment which may be used to evaluate student competencies across the range of modalities and multiple representations that students are now expected to be conversant with in becoming scientifically literate. This paper examines an example of how one NSW school participating in the ARC Discovery Project has begun taking a more innovative approach to assessment in Biology, by implementing multimodal assessment formats as part of the school’s science program. We include in this presentation an analysis of data which exemplifies current practice, teacher comments on assessment from interview data and video documentation of viva voce student assessments. We consider the potential and limitations of current test constructs and assessment practices in senior Biology and raise some issues in relation to whether existing assessment structures are sustainable in the climate of rapidly shifting representations of knowledge.

3

Introduction The Office of the Board of Studies (BoS) is the only statutory authority of the NSW government charged with curriculum and examination responsibilities. It prepares all K-12 school syllabi and the two external examinations – the School Certificate (SC for Year 10 students at Stage 5) and the Higher School Certificate (HSC for Year 12 students at Stage 6). The HSC examination is recognised nationally and internationally as an educational standard of excellence for secondary education. It is used by students to gain entry into further studies and into employment. The Stage 6 NSW HSC Biology syllabus is one of five HSC science course taught in the last 2 years of secondary schooling. The first year, known as the Preliminary Course covers evolution and ecology within an Australian context. The second year, known as the HSC course builds on the previous year and covers topics such as human physiology, disease and genetics along with optional modules on communication, biotechnology, human evolution, advanced genetics and biochemistry. The current syllabus was developed in the late 1990s, first implemented in 2000 and examined in 2001, with amendments made in 2002. There have been no substantial changes to the syllabus per se nor its examination since 2000. The aims, objectives and outcomes, and content have remained unchanged. Biology Syllabus – rationale and organisation The HSC Biology Course provides students with a contemporary view of the biology including a holistic understanding of concepts which explain the function, origin and evolution of life on Earth. The syllabus explores life at all levels of organisation – molecular to macro level, along with their interaction with and between other organisms and the environment. It regards evolution as the source of both unity and diversity. Humans are part of nature, often dominate ecosystems and have influenced the success and demise of many species. As such, people have a responsibility “to conserve, protect, maintain and improve the quality of all environments for future generations” (p. 6). The course develops students’ understanding of the interdisciplinary nature of science, the importance of research to extend to boundaries of current knowledge/understanding, the importance and value of experimentation to develop and test theories, the tentative nature of scientific explanations along with the “evidence and ideas and impact of science on society” (p.6). The practice of biology for HSC biology students involves practical work in the field, with ICT and in the school laboratory with/without the help/involvement of peers. The syllabus expects students to be able to solve a variety of problems and to communicate their knowledge and understandings in a variety of ways. The syllabus builds on work from previous years and assumes that students have “a substantial achievement level based on the science Stage 4-5 course performance descriptions”, whilst recognising various student interests and commitment to studying biology. At the end of the course, students will have developed knowledge and understanding of the following objectives (p. 8):

4

1. the history of biology 2. the nature and practice of biology 3. applications and uses of biology 4. the implications of biology for society and the environment 5. current issues, research and developments in biology 6. cell ultrastructure and processes 7. biological diversity 8. environmental interactions 9. mechanisms of inheritance 10. biological evolution.

And the following skills:

11. planning investigations 12. conducting investigations 13. communicating information and understanding 14. developing scientific thinking and problem-solving techniques 15. working individually and in teams.

Furthermore, students will develop positive values about and attitudes towards:

16. themselves, others, learning as a lifelong process, biology and the environment. The HSC Biology syllabus (p.9) also states that students are to have practical experiences (termed first and second hand investigations) which include:

undertaking laboratory experiments, including the use of appropriate computer-based technologies

fieldwork research, using a wide range of sources, including print materials, the Interne and digital

technologies using computer simulations for modelling or manipulating data using and reorganising secondary data extracting and reorganising information in the form of flow charts, tables, graphs, diagrams,

prose and keys using animation, video and film resources to capture/obtain information not available in other

forms. The Preliminary course (120 hours) taken by students in Year 11 incorporates the study of:

A Local Ecosystem (25 indicative hours) Patterns in Nature (35 indicative hours) Life on Earth (30 indicative hours) Evolution of Australian Biota (30 indicative hours)

The HSC course builds upon the Preliminary course. It incorporates the study of:

a) the core, which constitutes 90 indicative hours and includes: Maintaining a Balance (30 indicative hours) Blueprint of Life (30 indicative hours) The Search for Better Health (30 indicative hours)

b) ONE option, which constitutes 30 indicative hours and may comprise any one of the following: Communication Biotechnology Genetics: The Code Broken? The Human Story Biochemistry

5

Data released by the Board of Studies, indicates that of the 14,500 candidates who present themselves for examination each year, 60% have studied the Communication option and 20% Genetics: The Code Broken option. The Human Story is done by about 10% of students, with less than 5% doing Biotechnology. Very few students answer questions in the Biochemistry option. Some students do not attempt this section of the examination or answer more than one option. New Life Science (NLS) content within the NSW HSC Biology Syllabus As detailed in previous papers in this symposium, NLS is a rapidly expanding area of research integrating knowledge and understanding within the various science disciplines with educational, commercial, industrial and social applications. NLS includes areas such as biotechnology, genetic engineering and cloning. Research evidence continues to confirm that students are interested in biology that directly affects them now and in the future as informed citizens. Students are interested in controversial biological issues that have a moral and ethical basis with research indicating that lessons where these issues are addressed within the context of biological evidence enhance student learning of biology and skills in making ethical judgements. Within the NSW HSC biology syllabus there is potential for teachers to incorporate NLS without compromising either syllabus coverage or HSC examination success. Recent federal and state funding for ICT in schools, has given teachers opportunities to incorporate the Internet and other digital resources into their classroom practice. There are now high quality websites containing up to date biology, often developed by government agencies and universities, specifically geared for high school biology. These sites contain learning objects such animations, video clips, electron micrographs, practical demonstrations which might not be possible in classrooms and novel assessment tasks to name a few. For example, the Genetic Science Learning Centre of the University of Utah ( http://learn.genetics.utah.edu ) contains a wealth of information and digital imagery suitable for all the genetics within the HSC biology syllabus and extension material for teachers and the more able students. The site contains a polymerase chain reaction (PCR) virtual lab not possible in school labs that is simple to use by students. A selection of areas in the syllabus where NLS is found below, with further details available at: http://www.boardofstudies.nsw.edu.au/syllabus_hsc/syllabus2000_listb.html

6

Blueprint of Life (p.45) Chromosomal structure provides the key to inheritance

describe the chemical nature of chromosomes and genes identify that DNA is a double-stranded molecule twisted into a helix with each strand

comprised of a sugar-phosphate backbone and attached bases – adenine (A), thymine (T), cytosine (C) and guanine (G) – connected to a complementary strand by pairing the bases, A-T and G-C

identify how the following current

reproductive technologies may alter the genetic composition of a population:

Artificial insemination Artificial pollination Cloning

process information from secondary sources to describe a methodology used in cloning

outline the processes used to produce transgenic species and include examples of this process and reasons for its use

Current reproductive technologies and genetic engineering have the potential to alter the path of evolution

discuss the potential impact of the use of reproduction technologies on the genetic diversity of species using a named plant and animal example that have been genetically altered

analyse information from secondary sources to identify examples of the use of transgenic species and use available evidence to debate the ethical issues arising from the development and use of transgenic species

The Search for Better Health (p. 49)

identify the components of the immune response: antibodies, Tcells, B cells

MacFarlane Burnet’s work in the middle of the twentieth century contributed to a better understanding of the immune response and the effectiveness of immunisation programs

describe and explain the immune response in the human body in terms of: − interaction between B and T lymphocytes − the mechanisms that allow interaction between B and T lymphocytes − the range of T lymphocyte types and the difference in their roles

Biotechnology (sample from p. 58) Ethical issues related to biotechnology are considered in the decision-making processes

explain why different groups in society may have different views about the use of DNA technology

identify and evaluate ethical issues related to one of the following – development of genetically modified organisms (GMOs), animal cloning and gene cloning

use available evidence to identify and discuss ethical and social issues associated with the use of biotechnology

Genetics: The Code Broken (sample from pp. 60-61) Mechanisms of genetic change

Outline the ability of DNA to repair itself Describe the way in which transposable genetic elements operate and discuss their

impact on the genome The Human Genome Project is attempting to identify the position of genes on chromosomes through whole genome sequencing

Outline the procedure to produce recombinant DNA Explain how the use of recombinant DNA technology can identify the position of a

gene on a chromosome

7

Assessment of student learning A standards-referenced approach is used by the BoS at the HSC examination with schools reporting student achievement in line with BoS requirements for internal assessment. All schools must develop a program for internal assessment for each subject and ensure that the program makes explicit to students/parents the assessment tasks and their weightings for each task and dates/times these will occur in the school year. The internal assessment mark provided for each student for each subject by the school is based only on the HSC course (except for Mathematics) and must incorporate the specific mandatory requirements located in each syllabus document. The essential features of the standards-reference approach of the BoS are that: Student achievement is assessed and reported with reference to specified standards of performance Marks awarded to students reflect the standards they have achieved Comparisons can be made between students based on their achievement of standards Final examination marks are determined by the proportions of students who achieve each

performance standard. There is no predetermined pattern of marks. This means over time, while standards remain constant, the proportion of students achieving each standard may change from year to year.

There are no limits on the number of students who can reach the top standard. All students who meet the minimum standard receive a mark of 50. Students who perform above the

minimum standard expected receive higher marks. Reporting includes information on the knowledge, skills and understanding typically demonstrated

by students who achieve a performance standard. (http://www.boardofstudies.nsw.edu.au/syllabus_hsc/newhsc_assessment)

Examination marks and internal school assessment marks are moderated for each student so that no one student is disadvantaged. Both sets of marks are aligned with the same standard. Teachers are now well placed in terms of standards and student performance. The BoS suggests that teachers make certain that assessment tasks focus on outcomes listed in the syllabus, a range of tasks be used, students understand the outcomes that are assessed and why, marking schemes for tasks use the words of the outcomes and the performance bands to report achievement, and feedback to students is based on the marking scheme which indicates the outcomes achieved. The assessment task should provide students with opportunities for them to demonstrate what they understand and can do. Standards packages and web-based resources have been made available by the BoS. For example, the 2001 NSW HSC standards package for biology is available on CDROM and that for 2003 is on the BoS website. The following tables provide guidance for teachers in the organisation and management of school based biology assessment tasks. Table 1 shows the school assessment program from the HSC Biology Syllabus (p.80). Teachers are advised of the need to balance knowledge and understanding of content (H1-H10) with the skills outcomes (H11-16) (p.16). Table 2 provides a summary of internal and external assessment (p.82).

8

Table 1. School assessment instruments for HSC Biology Component Weighting Tasks could include: Maintaining a Balance 25 Blueprint of Life 25 The Search for Better Health 25 Option o Communication o Biotechnology o Genetics: The Code Broken? o The Human Story o Biochemistry

25

Assignments Fieldwork studies and reports Model making Open-ended investigations Oral reports Practical tests Research projects Reports Topic tests and examinations Note: • No more than 50% weighting may be allocated to examinations and topic tests • A minimum of 30% weighting must be allocated to tasks that assess students’ abilities to conduct firsthand investigations and communicate information and understandings based on these investigations

Marks 100 Table 2. Summary of internal and external assessment Internal Assessment Weighting

External Assessment Weighting

Core Modules Option Note: Assessment of knowledge, understanding and skills developed through conducting firsthand investigations should be incorporated into the core and option as appropriate.

75 25

A written examination paper consisting of: Core Modules Multiple-choice questions Short-answer questions Longer answer question/s Option Short-answer questions Longer answer questions

75 25

Marks 100 Marks 100 Whilst details of assessment tasks are specific to schools, most teachers set between 4-5 tasks throughout the HSC year. The last task is generally the HSC trial examination. These assessment tasks determine the final assessment mark provided by the school to the Board of Studies and are in addition to class tests and other forms of formal/informal assessment. Table 3 provides a format for developing an assessment program for Biology. Further information is located at: http://www.boardofstudies.nsw.edu.au/syllabus_hsc/syllabus2000_listb.html

9

Table 3. Developing an assessment program for Biology

Task 1 Task 2 Task 3 Task 4 Task 5 28 Nov 2 Mar 2 May 19 June 14 Aug

Outcomes H1-15

Modules/ Option

Weighting (syllabus)

Essay Oral Class prac test

Research Trial HSC

various 9.2 30% 10% 10% 10% various 9.3 20% 10% 5% various 9.4 20% 10% various 9.6 30% 20% 10% Marks 100% 10% 10% 15% 30% 35% The HSC examination and multimodalities The HSC examination for Biology is a 3-hour paper on the HSC Course only. It is set by an examination committee comprising biology teachers and university academics and marked by experienced biology teachers all appointed via a selection process by the Board of Studies. The examination is in two sections. Section 1 (75 marks) is divided into Part A (15 marks) with 15 compulsory multiple choice questions (MCQ) all of equal value and covering the HSC core modules. Part B (60 marks) contains short-answer and extended response questions drawn from the HSC core modules. Marks vary per question. Section 11 Options (25 marks) contains five questions with one from each option, all of equal marks. There are several questions within each option and students are expected to attempt one question. The options questions are based on the HSC option list above. The Board of Studies makes available on its website all past examination papers, marking guidelines for each question and a report from the chief examiner/markers. A standards package for 2001/2003 is available for teachers and contains exemplars of student work at each of the performance bands. Examples of questions are as follows: 1. Multiple choice question (2002)

10

2. Short/extended response Question 18 (2006) (7 marks) a) Name ONE adaptation in an Australian terrestrial plant that assists in minimising water loss. (1) b) Explain why it is important for plant cells to control water loss. (2) c) Plant breeders have developed a new variety of terrestrial plant which has one structure that appears

to assist in water conservation in hot, dry environments. Design a first-hand investigation the plant breeder could use to determine if this structure assists in

water conservation. (4) Question 24 (2007) (6 marks)

(a) Why is it important to monitor oxygen levels in the blood during surgery? (1) (b) Explain ONE advantage of the T-Stat oximeter over the pulse oximeter? (2) (c) Explain TWO changes in the chemical composition of blood as it moves along a capillary. (3) Question 28 (2001) (8 marks) Evaluate the impact of major advances in scientific understanding and technology, in the field of genetics, on developments in reproductive technologies. A trawl through all past papers from 2001-2008 shows the following representations used for the modules and options: Text only questions with no stimulus material - MCQ, Q 18 above. Text as a stimulus – Q 28 above. Mathematics calculations Flowcharts – provided or candidates are asked to draw Graphs – provided or candidates are asked to draw Diagrams – provided or candidates are asked to draw Table of data - provided or candidates are asked to draw

11

The majority of HSC questions dealing with the New Life Sciences are found in the ‘Blueprint of Life’ core module and the options, ‘Genetics: Code Broken?’ and ‘Biotechnology’. Most of these questions are of varying complexity. They are often text only and/or with limited stimulus material. Questions involving diagrams and tables all concerned DNA structure/replication with nothing about translation of RNA/protein synthesis. A selection of these questions follows: Question 15 (2002)

Question 16 (2004) (4 marks)

12

Question 18 (2005) (4 marks)

Question 27 (2007) (3 marks) Construct a simple flowchart to describe the process of DNA replication.

Genetics: The Code Broken Various

• Evaluate the current use of gene cloning in animals and plants. (7marks) • Discuss how major advances in our knowledge of genetics have changed our understanding

of the way genes direct the structure, function and development of an organism. (7 marks) • Discuss the impact on the genome of transposable genetic elements. (4marks)

Biotechnology Various

• Draw and label a diagram showing the sequence of events that result in the formation of recombinant DNA.

• During your study of Biotechnology you gathered secondary information to identify that complementary DNA is produced by either reverse transcribing RNA or the Polymerase Chain Reaction.

(i) Describe how you processes and analysed the gathered information (4marks) (ii) State how you assessed the reliability of the data obtained. (1 mark)

13

Given the nature of NLS, its representation in multi-modal formats and its nexus with ICT, it is unfortunate that students are not provided with examination opportunities to showcase their understanding of its concepts other than in written text. This could easily be done with a little imagination. For example, in the recent 2008 examination, students were asked:

How could a mutation in DNA affect polypeptide production? How could a change in a polypeptide affect cell activity?

The concepts being examined here could equally have been assessed if the examiners had provided a model strand of DNA as a stimulus, asked students to make a complementary strand indicating a mutation (e.g. base substitution) and provided a table or an outline of a flowchart for students to complete indicating the correct/incorrect RNA, the triplets and polypeptide produced. The current research project has identified a small group of teachers who have made use of different modalities to assess their students’ understanding of protein synthesis – an understanding of which is essential to answer the above question. Multimodal assessment formats – a case-study of innovative practice This section of the paper examines how a metropolitan public school in NSW has begun to take a more innovative approach to assessment in HSC Biology by implementing multimodal assessment formats as part of the school’s science program. Context and description of task For the internal school assessment for the HSC in 2008, Year 12 Biology students undertook an oral examination as an assessment task for the unit ‘Blueprint of Life’. The assessment made use of two, three-dimensional models: first, a ball-and-stick model representing the physical structure of the DNA molecule, and second, a model of the biochemical processes of protein synthesis (Figure 1), constructed by a former student for assessment within the same unit of work in a previous year. Students were instructed to refer to the models in answering a series of questions on the topic of DNA and protein synthesis. The examiners also referred to specific parts of the model in their questions.

Figure 1. ‘Biochemical processes of protein synthesis’: A student-constructed model

14

The questions ranged from factual recall questions (e.g. 1a. What is the name for this type of molecular structure?) to explanations of complex concepts and processes (e.g. 3a. How does mRNA determine the sequence of amino acids in a polypeptide?). Examiners guided students in their answers by asking a series of sub-questions. The questions and sub-questions from the teachers’ assessment grid are displayed in Table 4. Where necessary, examiners provided further scaffolding for student responses by asking additional questions to facilitate more specific or extended responses. Table 4. Questions for oral assessment task – A model for protein synthesis

Questions Sub-questions 1. With reference to the model explain

the structure of DNA The DNA molecule is like a twisted ladder. a. What is the name for this type of molecular structure? b. What are the ‘rungs’ made out of? c. What makes up the sides of the ladder? d. Where are the ‘rungs’ attached to on the sides?

2. Describe the role of mRNA and tRNA in protein synthesis

a. What is the role of mRNA in protein synthesis? b. What is the name for this process? c. What is the role of tRNA in protein synthesis? d. What is the name for this process?

3. Explain what is meant by the DNA code for amino acids, in terms of nitrogenous base sequence or codons. Support your explanation of the code by quoting specific examples

DNA is used to code for amino acids- a. How does mRNA determine the sequence of amino

acids in a polypeptide? b. Can you give a specific example?

4. Mutations sometimes occur in DNA. Referring to your model, explain how a mutation may affect protein synthesis. Outline, using a specific example, how a mutation may: • affect biochemical processes • result in macroscopic changes

a. What is a mutation? b. How can a mutation affect protein synthesis? A mutation may affect some aspect of cellular activity or result in a macroscopic change in an organism; c. Can you give a specific example of such a mutation?

5. Outline the contribution of one of the following scientists to our understanding of the chemical and structural features of DNA - Watson / Crick / Franklin / Wilkins

a. What was the nature of the work that he/she did? b. What did he/she propose about the structure of

DNA? c. How did he/she come to that conclusion?

6. Mode Use of provided model and/or diagram 7. Tenor Communication skills and terminology use

15

Video recordings of the oral exam task for 8 consenting students included in the case study were analysed in terms of the structure of the interaction between the student and the teacher-examiner and the features of the exchanges (IRF moves) for each question in the task (adapting the categories outlined in Freebody, 2003, pp. 95, 96). A number of additional sub-categories were developed to further describe follow-up moves that functioned to encourage more precise or more complete answers from the students (i.e. FACILITATE, e.g. That process also has a name. Can you think of that name?). Student responses were also compared in terms of knowledge-cognitive dimensions (Anderson & Krathwohl, 2001) and the use of different modalities applying the frameworks developed for this project; and the use of key words and scientific language pertinent to the topic. The excerpts below highlight the contrasting features of two students’ performance on Question 2, “Describe the role of mRNA and tRNA in protein synthesis”. Exemplars of viva voce student assessments Student F1 struggles with the more demanding aspects of the task and has difficulty recalling some of the key scientific terms related to the topic (e.g. transcription, translation). Her frustration with articulating her knowledge of the topic is encapsulated in her words “I know it but I just don’t know how to say it”. While the student is able to use the physical models to identify the structures of a cell nucleus with relative ease, she fails to demonstrate higher order conceptual understanding of the functions of these components and the relationships among them. The link between conceptual understanding and the more complex processes represented in the second model (Fig. 1) begins to break down, as is illustrated in the following excerpt from a transcript of the recording. Excerpt 1. Student F1 - Question 2 (c & d)

59 T1 =Right^ (.) could^ you describe for me:e the role of tRNAv in protein synthesisv 60 S Um yep^ so the tRNA ((student clears her throat)) (.1) um (.3) so that’s (.) ((points to tRNA

strand in model)) that’s th:he tRNA that the amino acids// 61 T1 //Y:yeah good^= 62 S =A:and the codon/the:e anti-codon^= 63 T1 =Yeah^= 64 S =So:o um (.) what^ happens is (.1) tRNA’s floating around and has to match with (.1) a:a (.)

specific codon (.2) in order to creat:te um (.4)// 65 T1 //Well:l lets say [what’s sort of] ((gestures to encourage response)) 66 S [An amino chain] ((forms ‘chain’ gesture with hands)) 67 T1 Yeahv-OK it’s picking up (.) amino acids right^= 68 S =Yep= 69 T1 =OK and then:n what does it do:o with that (.) amino acidv (.) when its picked it up from the

cytoplasmv ((gestures to encourage response)) 70 S Um:m it (.) goes to the ribosome^= ((points to ribosome in model)) 71 T1 =Yup (.) and how does it bind to the ribosomev 72 S ((student takes a deep breath)) U:um ((students lets out the breath)) (.1) how does it bind to

the ribosome ((student talking to herself)) (.2) um:m I’m^ not sure// 73 T2 [Well you mentioned] 74 T1 [Wellv what^] are those three little things therev (.2) ((points to the model)) see^ (.)

you’ve got two amino acids (.) ((points to part of model)) there^= 75 S =Yeah=

16

76 T1 =Attached (.) there^ so (.) what^ are those three little things that are attaching the (.2) ah:h (.1) amino acid (.1) to the mRNA calledv

77 S U:um^ (.) does-this^ ah:h ((spoken letting out a deep breath)) (.) well they’re^ the:e (.2) they’re-the c:c (.) ((points to model)) so you’ve got two codons^=

78 T1 =OKv (.) alright^ (.) don’t^ (.1) don’t^ go any further that’s fine (.1) can-you:u name tha:at processv

79 S (.2) 80 T1 That’s occurring (.1) ((gestures towards model)) in the cytoplasm (.) where the tRNA is

taking amino acids to:o (.) the:e mRNAv (.) that process also has a name can-you (.) think^ of that name^

81 S (.2) Uh:h-uh ((shakes head)) 82 T1 OK^ (.) don’t worry^ (.) we won’t get stuck on it^ (.) we’ll:l come back to that later (.1)

((teacher takes a deep breath)) The examiner’s facilitating moves in this exchange guide the student towards a minimally acceptable answer. Much of the work in the interaction to generate an extended explanation from the student is performed by the teacher, with the student contributing key words and concepts to complete the picture. Consequently, much of the knowledge-cognitive work done by the student consists of the recall of facts and concepts with little hierarchical organisation of material. In contrast, Student M1 displays mastery of the subject matter and takes control of the interaction. The examiner’s questions act as pointers to guide the student’s responses and to some degree constrain his explanations to the answers dictated by the marking criteria. The student readily transfers his understanding of these complex concepts and processes to the unfamiliar model and successfully applies his knowledge to this new situation. He has a good grasp of the key terms specific to the topic and articulates his answers with ease, using the appropriate scientific language. Excerpt 2. Student M1 - Question 2 (a, b, c, d)

15 T1 OKv (.1) now mo^ving away from:m the actual structure^ (.) to:o the roles of the mRNA and tRNA in protein synthesisv (.1) could^ you describe for me^ (.) the role of the mRNA in protein synthesisv (.) and you can refer to the model therev

16 S Um well^ (.) I think this ((points to model)) is mRNA-(yeah) mRNA is coming through the nuclear pore which is in the hole there^= ((indicates pathway through hole in model))

17 T1 =Yeah= 18 S =And previous to that it (.) the:e DNA had just unwound and the mRNA had transcribed the

code um:m on the (d)RNA and then transferred-then leaves-the cell connect attaches to the ribosome where its starts to build up the ((inaudible))=

19 T1 OK (.) right an:nd (.) could you name that ((points towards model)) process^ that occurs in the nucleus for me^

20 S Um (.) transcriptionv 21 T1 Right (.) goodv (.) 22 T1 OKv (.) now (.) could you (.) explain it to me please what the role of tRNA is in protein

synthesisv 23 S tRNA is found is the cytoplasm and it enters the ribosome when the mRNA is in there (.) and

the:e um tRNA links onto the (.) as an anti-codon links on to the corresponding amino acid chain to/for-the (.) to build-up-the (.) polypeptides^=

24 T1 =Yep (.1) ah:h (.) OK:K^ (.) could you just^ expla:ain:n (.2) to me (.) how the:e (.) tRNA binds to the mRNAv

25 S The (.) tRNA binds to the mRNA by their um corresponding nuc/net nucleotides or um nitrogen-spread nitrogen bases um binding together like (.) adenine binding to uracil=

17

26 T1 =Yeah:h and these areas have a specific name could-you (.2) 27 S The codon binds with the anti-codon// 28 T1 //OK that’s it (.) that’s what I wanted thank you^ (.1) 29 OK^ and that process is called^ 30 S Translation^

(Refer to Appendix for transcription conventions.)

18

Features of the interactions A comparison of the exchanges for Student F1 and Student M1 reveals that although the examiner asked exactly the same number questions in the elicitation moves (N=16), there was a very substantial difference in the number of follow-up moves the teacher used to facilitate student responses (42 for Student F1, 16 for Student M1). Table 5 displays the results for the entire assessment interaction for each student, as well as for Question 2 discussed in the examples above.

Table 5. Length of exchanges

STUDENT F1 STUDENT M1 NO. OF MOVES TOTAL (Q2) TOTAL (Q2) Total 218 (63) 113 (20) T initiate: elicit 16 (4) 16 (4) T follow-up: facilitate 42 (16) 16 (2) S respond: answer 46 (13) 29 (7)

The effort involved in the interaction, while it maximised the opportunities for Student F1 to demonstrate her understanding and supported her recall of information, did not advantage her performance on the task in any demonstrable way. The answers of the two students differed in the proportion of factual and conceptual knowledge demonstrated, as well as the type of cognition involved (Table 6), with Student M1 displaying higher levels of conceptual understanding and Student F1 demonstrating a higher proportion of factual knowledge. (See Appendix Table A for further detail on the knowledge-cognitive dimensions).

Table 6. Student answers: knowledge-cognitive dimension

STUDENT F1 STUDENT M1 KNOWLEDGE-TYPE1 N=46 N=29 factual 58.70% 27 48.28% 14 conceptual 41.30% 19 51.72% 15 COGNITIVE-TYPE2 N=46 N=29 remember 39.13% 18 20.69% 6 understand 32.61% 15 51.72% 15 apply 28.26% 13 27.59% 8

In relation to the cognitive dimensions of the tasks, there were no pronounced differences between the students in demonstrating factual knowledge (Table 7). However, the differences become notably marked where conceptual knowledge is concerned (Table 8). For Student M1, 73.33% of responses were found to be in the category ‘conceptual-understand’ while only less than half (47.37%) of Student F1’s

1 ‘Procedural’ and ‘metacognitive’ knowledge types were not a feature of this assessment task. 2 The questions in this task did not elicit cognitive response types ‘analyse’, ‘evaluate’ or ‘create’.

19

answers were in this category; another 47.37% of responses for F1 fell into the category of ‘conceptual-remember’.

Table 7. Student answers: factual knowledge

STUDENT F1 STUDENT M1 KNOWLEDGE-TYPE factual N=27 N=14 COGNITIVE-TYPE remember 33.33% 9 28.57% 4 understand 22.22% 6 28.57% 4 apply 44.44% 12 42.86% 6

Table 8. Student answers: conceptual knowledge

STUDENT F1 STUDENT M1 KNOWLEDGE-TYPE conceptual N=19 N=15 COGNITIVE-TYPE remember 47.37% 9 13.33% 2 understand 47.37% 9 73.33% 11 apply 5.26% 1 13.33% 2

Multimodal representations and knowledge Consistent with the findings for the cognitive dimension ‘apply’, there was greater use of multiple modalities for Student F1 than Student M1, both in formulating answers to various aspects of the teacher’s questions and in the teacher’s facilitation of the students’ responses (Table 9). While reference to the models was crucial for successful performance on Questions 1 and 2, the remaining questions (3 to 5) did not depend on their use. Where verbal communication alone was inadequate to articulate one’s ideas, the increased use of gesture and reference to the model for clues could be seen as an attempt to complement the effort.

Table 9. Multimodality

STUDENT F1 STUDENT M1 Feature Percent N Percent N SPEAKER N=40 N=17 teacher 62.50% 25 52.94% 9 student 37.50% 15 47.06% 8 EXCHANGE N=40 N=17 introduction 5.00% 2 5.88% 1 question-1 30.00% 12 58.82% 10 question-2 47.50% 19 35.29% 6 question-3 12.50% 5 0.00% 0 question-4 5.00% 2 0.00% 0

20

question-5 0.00% 0 0.00% 0 Q1-PART N=11 N=10 1a 18.18% 2 20.00% 2 1b 27.27% 3 20.00% 2 1c 18.18% 2 60.00% 6 1d 36.36% 4 0.00% 0 Q2-PART N=17 N=6 2a 41.18% 7 50.00% 3 2b 5.88% 1 33.33% 2 2c 41.18% 7 16.67% 1 2d 11.76% 2 0.00% 0 MOVE-TYPE N=38 N=17 frame 5.26% 2 5.88% 1 initiate 18.42% 7 35.29% 6 respond 39.47% 15 47.06% 8 follow-up 36.84% 14 11.76% 2 MM-1 N=40 N=17 spoken 100.00% 40 100.00% 17 written 0.00% 0 0.00% 0 MM-2 N=37 N=14 gestural 97.30% 36 100.00% 14 kinaesthetic 2.70% 1 0.00% 0 MM-3 N=34 N=17 material 100.00% 34 100.00% 17 operational 0.00% 0 0.00% 0

Discussion of results

The introduction of physical models to the assessment adds another cognitive dimension to the task – that of application or the transfer of knowledge. Thus, not only are students required to communicate orally their knowledge and understanding of scientific facts, concepts and processes, they are also required to transfer their conceptual understanding to new representations of these complex concepts and relationships. However, without prior knowledge and understanding of the biology concepts, the model did not help students to answer the questions as is evident in the case of Student F1. Students must firstly be able to identify the structures represented; and while the models may display some of the relationships among the structures spatially and perhaps provide cues about the sites and sequence in processes such as transcription and translation, students must be able to interpret these in relation to what they already know about the function of these components - they cannot rely on the model for the answers. One advantage of the assessment format described in this section of the paper is that it has the potential to reveal the breadth and depth of student knowledge and understanding of a given topic. The oral format enables students who are more articulate in speech than in writing to demonstrate their mastery of a topic; in such cases, a written paper may not permit an accurate assessment of their understandings:

“There's some students in my class who, from what they can tell me in class ought to be band six kids but their marks don’t show that because they have to write their answers in written forms.” (Teacher BT)

21

Another advantage of the viva voce situation is that potential confusion about the expectations of an exam task can be addressed immediately, allowing the student to proceed with demonstrating what they actually know about a topic. The following comment from a teacher interview accentuates this problem in assessment:

"... There was a question in the trial exam, they had 30 base pairs and said how many codons does this make, so 10 was one of the answers and eight was another, and this student put eight and said … ‘I’m not going to count the start codon and I’m not going to count the stop codon so that makes just eight amino acids in the middle’, ... so he was using his extension knowledge and of course got the question wrong. And so that’s quite frustrating for him and it’s a very bright student as well…” (Teacher LB)

The limitations of the assessment format are not dissimilar to those experienced in other testing situations – the demonstration of student knowledge is constrained by the design of the exam task, the pressures of the assessment situation, time allocation per student, and in this case, a new and unfamiliar type of assessment task. It should be noted that what drove the interactions in this instance was the elicitation of correct answers within a limited timeframe. For students such as M1, who arrived at the required response relatively quickly, further opportunities to demonstrate a more sophisticated or comprehensive understanding of the topic were curtailed, while a concerted effort was made to allow students such as F1 an opportunity for success. Concluding remarks The assessment structures currently applied in the HSC Biology examination provide a number of options for assessing student understanding of scientific knowledge and skills in the subject. While the formal external examination is restricted to written responses which are suited to the display of certain types of knowledge, the internal school assessment permits a greater range of testing formats which have the potential to: a. reveal the breadth of student knowledge and depth of understanding of a topic; b. take into account the kinds of representations that are best suited to specialised

topics (e.g. New Life Sciences); and, c. provide a greater range of assessment task types for students who are more

articulate in modes other than the written word. In a climate where representations of scientific knowledge are rapidly being transformed by digital technologies, the way in which students are assessed for their understanding and application of this knowledge needs to be revisited. References Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A

revision of Bloom's Taxonomy of Educational Objectives. New York: Longman. Freebody, P. (2003). Qualitative research in education: Interaction and practice. London: Sage

Publications. Office of the Board of Studies – for HSC syllabus, support and examination documents

22

http://www.boardofstudies.nsw.edu.au APPENDIX Transcription Conventions (adapted from Freebody, 2003, pp.98-99) / latched turns (no intervening pause)

// heard as interruption

[ said simultaneously; beginning of overlapping talk

] end of overlapping talk

= link connects ‘latched’ or closely connected utterances

co:old extended vowel or consonant

(.4) approximate length of pause in seconds

(.) pause of less than a second

(( )) transcriber’s comment; non-verbal features of talk

(mouse) uncertain transcription

( ) untranscribable talk

h, hh aspirant sound

so-he-is words said very quickly that run together

along words read from text

^ interrogative or rising intonation v falling intonation

must emphasis

T teacher

S(s) student(s)

23

Table A. Student Knowledge and Cognitive Dimensions Factual (Terminology, details, and other basic elements of biology; i.e. micro elements)

Conceptual (Complex processes, generalisations, theories, and models; i.e. interactions of micro elements that make macro phenomena)

Remember: Student reviews facts and details of past lesson, refers to prior knowledge, or states terminology.

e.g. When T asks if anyone knows the word to describe where something is found, S replies “Distribution”.

Understand: Student demonstrates comprehension of simple facts and details via explanation, interpretation, summarisation, classification, and other mental manipulations.

e.g. S asks teacher a coherent follow-up question about the size of an onion cell.

Apply: Student transfers knowledge of facts, and details to a new situation.

e.g. S uses knowledge of a simple chemical reaction to combine two chemicals and elicit the reaction.

Analyse: Student identifies and examines the connections between different facts and details (distinct from conceptual/analyse as it is a bottom up analysis)

e.g. S orders organisms according to the size of their skeletal system.

Evaluate: Student questions or make conclusions relating to the accuracy of facts and details.

Create: Student restructures or extends knowledge of facts and details to produce something new and original.

e.g. S combines facts they have learnt about the mosquito into a poster

Remember: Student reviews previously learnt complex concepts.

e.g. S recalls and describes how photosynthesis works

Understand: Student demonstrates comprehension of complex concepts via explanation, exemplification, summarisation, and other mental manipulations.

e.g. S verbally interprets a figure depicting a complex biological model

Apply: Student transfers knowledge of complex processes, theories, and models to a new situation.

e.g. S uses knowledge of the heart to dissect a cow heart

Analyse: Student identifies and examines the connections between components of a complex model, process, or theory (distinct from factual/analyse as it is a top down analysis)

e.g. S identifies and draws links between the micro and macro levels of a system, from the organism level to the organ level down to the molecular level.

Evaluate: Student appraises the value of a conceptual scheme, model, or complex notion or process.

e.g. S assesses which of two models best represents a particular phenomenon

Create: Student restructures of extends knowledge of concepts and complex processes to produce something new and original. e.g. S designs an experiment with original hypothesis designed to extend knowledge of a particular concept e.g. S writes an essay about a complex model

Derived from Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's Taxonomy of Educational Objectives. New York: Longman. Remember: Retrieving knowledge from memory via recognition and recall; rote learning. Requires the

repetition of material. Understand: Determining the meaning of instructional messages. Requires the mental manipulation of

‘whole’ material (explaining, interpreting, exemplifying, paraphrasing, etc.). Apply: Using learned material in new situations. Requires the concrete implementation or

execution of knowledge (via models, presentations, interviews, etc.). Analyse: Breaking material into its constituent parts and determining how the parts relate to one

another, to the overall structure, or to external material. Requires critical or ‘higher order’ thinking.

Evaluate: Making judgments about merit or value, based on given criteria and standards and often without a correct answer. Requires checking or critiquing; may include recommendations for improvement.

Create: Using old knowledge to plan or generate a product, pattern, or structure that is new and original. Requires parts to be put together in a new way or synthesised into something different.