SABER 2017 Abstracts- Short Talks

SABER- Abstracts- Short Talks

FRIDAY - ModelingDeciphering High School Biology Modeling Instruction Practice: A Comparative Case StudyFeng Li*, Florida International University; Eric Brewe, Drexel University; Zahra Hazari, Florida International University; George E. O'Brien, Florida International University; Haiying Long, Florida International University[abstract # 174]High school biology Modeling Instruction (MI) curriculum has been under development for about a decade. Substantial effort has been made in developing biology MI materials and integrating MI into high school biology classes. Attention has yet been paid to monitoring, evaluating, and promoting high school biology instructors’ teaching practices with MI curriculum. The current study focused on teaching practices of high school biology teachers from Miami-Dade County Public Schools (MDCPS), who have participated in a Modeling Workshop and utilized the MI curriculum for two years. In the lens of modeling theory of instruction, we evaluated how well MI was practiced in a high school Biology I class in MDCPS, by comparing it with a non-MI Biology I class in the same school. Both classrooms were observed during their teaching period of evolution for three weeks. The Biology Identity and Persistence Survey was administrated at the beginning and the end of the data collection period in both classes to statistically assess students’ biology identity, evolution identity, and career aspirations. Five students from the MI class and four students from the non-MI class were interviewed respectively about their perceptions of their biology teachers’ teaching practices after the instruction of evolution had completed. We employed a constant comparative approach with the qualitative data and regression with the quantitative data to decipher and compare the teaching practices of the MI biology teacher with the non-MI biology teacher. Quantitative results demonstrated no statistically significant differences between the MI and non-MI classes in students’ biology identity and evolution identity development, and their career aspirations. Findings of the qualitative analysis indicated that, compared to the non-MI instructor, the MI instructor did not provide students with significantly more autonomy in designing, validating, implementing, and revising models. Although students worked in groups with hands-on activities, which happened in both MI and non-MI classes, it was still a teacher-centered environment in both classes. Compared to those in the non-MI class, students in the MI class also followed the teacher’s instruction in every activity with no significant autonomy, and no significantly more meaningful interactions occurred among students. As the result, no significant performing biology identity was observed in both MI and non-MI classes. In conclusion, it is critical to not only employ MI activities with students, but also efficiently engage students with autonomy and encourage students’ interactions to benefit from the biology MI curriculum.

Persistence of non-canonical ideas following in-class model development by groups of students.Becky Matz, Michigan State University; Kristin Parent, Michigan State University; Andrea Bierema, Michigan State University; Jon Stoltzfus*, Michigan State University[abstract # 161]Current reforms in STEM education emphasize developing scientific skills and understanding of core ideas. We implemented modeling activities aimed at meeting both these needs in a large-enrollment introductory majors biology course. Groups of three students use ideas from previous lectures to develop a model explaining a related phenomenon with the goals of fostering models as sense-making tools and reinforcing core ideas. We previously analyzed recordings of groups developing models and found that students work to make sense of important aspects of the phenomenon. As students develop explanations, however, they occasionally settle on non-canonical ideas. While the development of alternative explanations is a desirable aspect of scientific modeling, long-term persistence of these ideas is not desirable. In this study, we focus on one activity that uses ideas about how cells activate genetic information via signal transduction to explain why normal cells only divide when the proper signal is present, why mutations can lead to uncontrolled cell division, and how a drug targeting Ras could help treat cancer. We compare students' ideas from models and isomorphic exam problems to address three research questions. 1) Do non-canonical ideas incorporated into models persist when answering exam problems? 2) Is there a relationship between academic preparedness, gender, or race/ethnicity and persistence of ideas from the model on the exam? 3) Is there a relationship between group composition and persistence of ideas from the model on the exam? Iterative cycles of inductive coding revealed four mechanistic aspects of Ras function commonly found

1

1

SABER 2017 Abstracts- Short Talks

in student models that varied in agreement with canonical scientific ideas. We constructed isomorphic exam problems aimed at testing these ideas, tested coding until we achieved interrater reliability, and coded both models and exams. Chi-square analysis for goodness of fit revealed that students in groups with non-canonical ideas in their model are more likely to include canonical ideas when answering exam problems than to retain non-canonical ideas. Pearson's chi-square test of independence revealed no meaningful relationships with respect to gender or race/ethnicity but showed students with lower academic preparedness less frequently retain some canonical ideas from the model when answering exam problems. Finally, multinomial regression revealed effects of group composition and individual student characteristics related to academic preparedness. Based on these data we conclude that students are able to overcome incorporation of non-canonical ideas into their models but this is influenced by academic preparedness and group characteristics

Context Increases Knowledge Integration in Student’s Genotype-to-Phenotype Models in Molecular GeneticsKristy Wilson*, Marian university; Allison Chatterjee, Marian University[abstract # 166]Students often see biology and the life sciences as disconnected facts. Research has shown that the building of models like concepts maps and their variations like Structure Relationship Function (SMRF) box and arrow models can allow students to see the big picture. The specific question addressed is: would models better instill system understanding and skill transfer if they were generalized or if they incorporated a specific biological example to create context. Students were asked to draw SMRF models in response to questions that provided a list of structures that would be in boxes to describe an overall function in two sections of a Molecular Genetics course for biology majors. One section received instructions that included an interesting example gene or situation that were related to issues like human disease, treatment, or the environment. Whereas, the other section had the same overall function but lacked the example and instead made their SMRF model about a numbered or lettered gene and phenotype. SMRF model questions were given as practice throughout the course and collected during high stakes assessments. The demographics for both sections of the course were similar in terms of average incoming SAT score, gender, and ethnicity. SMRF models were assessed on formatting, expression of the function (contextualization), and on correctness. Analysis reveals that over the course of the semester the students receiving generalized questions had an decreased score, in contrast to the students receiving a specific example score increased over the semester on the SMRF models. A linear regression was performed to assess the change in SMRF model score over the semester and their was a statistically significant different between the slopes. This indicates that providing a specific example allows students to better integrate topics across the course. Each of the SMRF questions was analyzed for the number of physical levels that the questions contained. In addition, students were assigned an expansive SMRF model that asked them to integrate ideas from the entire course. This was analyzed for model complexity using web causality index (WCI) and for number of topics across the course that the model integrated together. The presentation will discuss two competing theories to define student performance and understanding: cognitive load and motivation. We conclude that providing context (especially for novice learners) is important when modeling and integrating understanding.

Improving student tracing through metabolic systems through a computational model-based learning moduleLara Appleby*, University of Nebraska-Lincoln; Heather Bergan-Roller, University of Nebraska-Lincoln; Nicholas Galt, Valley City State University; Joe Dauer, University of Nebraska-Lincoln; Tomas Helikar, University of Nebraska-Lincoln[abstract # 51]Many topics common to introductory life science courses can be seen as systems, or networks of components related to each other by rules. This kind of systems perspective can benefit students because through it they may be able to see the similarities among disparate topics and thereby approach those topics via common reasoning strategies, including logic. Computational tools for addressing systems-related learning objectives are growing in number and accessibility, and evaluation of the efficacy of related lessons could help move such lessons into more classrooms. We assessed the efficacy of a computational model-based lesson that takes a systems perspective of cellular respiration. The lesson included (1) instruction in the systems-related concepts of stock and flow relationships and negative feedback and (2) opportunities to practice tracing through a system to explain a phenomenon. An assessment administered before and after the lesson probed students’ declarative knowledge and systems thinking skills, including the ability to recognize examples of one of the two main types of components

2

2

SABER 2017 Abstracts- Short Talks

within the cellular respiration system, apply knowledge of the interconnectedness of components within the system, and trace through a system including alternate pathways. The module was implemented at a large public research university during two semesters (total n = 649) during a recitation session of an introductory life sciences course for majors. Participants improved in their performance on an assessment of systems-related learning objectives from before to after the lesson (Wilcoxon V = 70084, p < 0.001). Demographic variables including gender and first-generation status were poor predictors of growth in the lesson, suggesting that the lesson was equally accessible across demographic groups. To further characterize student learning of systems concepts in the context of cellular respiration, we also analyzed student-edited network diagrams and responses to short answer prompts. These analyses highlighted certain parts of the cellular respiration system and certain ideas in general systems thinking as more difficult than others. Together, the results reinforce the need for systems-thinking related lessons and highlight computational models as potential contributors to their efficacy.

Computational modeling: a tool to improve student reasoning with biological systems.Tomas Helikar*, University of Nebraska-Lincoln; Lara Appleby, University of Nebraska-Lincoln; Heather Bergan-Roller, University of Nebraska-Lincoln; Nicholas Galt, Valley City State University; Joe Dauer, University of Nebraska-Lincoln[abstract # 56]Keywords: Models and Modeling, Assessment of student learning and instructional innovation Biological processes at almost every scale of biological organization are governed by complex, non-linear networks. Mathematical modeling and computer simulations have emerged as integral to life sciences research to understand these biological processes. Given the shift in life science research it is important for biology education to evolve in order to equip our students with skills to reason conceptually, mechanistically, and quantitatively, and to answer emerging life science questions. To address these challenges, we developed a new simulation- and model-based approach and software, Cell Collective, to learning about biological processes. This method enables students to learn about biological processes by creating, simulating, and interpreting computational models of complex systems. Results from the deployment of this approach within the laboratory portion of introductory biology courses over the past three years indicate improvement in short and long-term learning, and provide insights into student modeling practices. Results from short-term efficacy pre-post assessment (n=471) of a lesson about cellular respiration with a focus on intracellular conservation of matter suggest that the lesson improved students’ declarative knowledge and systems thinking skills (ability to distinguish molecules from processes, ability to distinguish among different types of stocks and flows, and general ability to trace matter through a metabolic network; Wilcoxon V = 70084, p < 0.001). Long-term efficacy was assessed using lecture exam performance collected over three semesters (n=1043) with five different instructors. Students who engaged in our lesson answered significantly more questions correctly (74%) than an equivalent student (according to composite ACT score) who used a different simulation software (71%, effect size = 0.05, p < 0.05). Efficacy of different modeling practices was analyzed using a simulation lesson about gene regulation (Lac operon) in two formats (model interpreting vs. model constructing) over two semesters. Results from the assessment of students’ pre-post conceptual models (n = 276) indicate that students who constructed a computational model of the Lac operon drew smaller (p<0.05, ES=0.27), higher quality (p<0.05, ES=0.7) post-lesson conceptual models than students who investigated a pre-constructed model. Even single, short-term computational modeling using this software can be effective in improving student learning of complex biological systems. Further development of computational modeling lessons on glucose homeostasis, food webs, cell cycle, and feedback loops will provide instructors with options to implement modeling activities throughout their courses. Collectively, these activities will further improve students’ systems thinking and prepare them for 21st Century biology.

FRIDAY - AssessmentHow question types reveal student thinking: An experimental comparison of multiple-true-false and free-response formatsBrian Couch*, University of Nebraska-Lincoln; Joanna Hubbard, University of Nebraska-Lincoln[abstract # 110]Assessment instruments represent a fundamental component of any course as they allow students to interact with material, provide instructors and students with information regarding student thinking, and produce scores

3

3

SABER 2017 Abstracts- Short Talks

reflecting student performance. When choosing an instrument format, instructors face the task of weighing the ability of different formats to assess student thinking against the time and resources required to develop, administer, and score the assessments. To maximize the impact of these instruments, instructors must understand how student results can be affected by different question formats. In this study, we sought to compare how student responses differ between similar questions posed in the closed-ended, multiple-true-false (MTF) format and the open-ended, free-response (FR) format. To compare responses to MTF and FR questions, we developed experimental question stems that could appear almost identically in either the MTF or FR formats. The true-false statements following the MTF question stem were generated from correct and incorrect student responses to open-ended questions from previous years. Coding rubrics were developed for FR answers to determine the extent to which students included correct, incorrect, or unclear responses related to the four ideas targeted in the matched MTF questions. Thirty-two experimental MTFxFR questions were strategically embedded across different versions of course exams, so that we could compare how similar groups of students performed on questions posed in either format. By analyzing and comparing student responses in the two formats, we found that correct, but not incorrect, response rates correlated between MTF and FR questions. We also discovered that the ratio of correct to incorrect responses differed between these formats and that lower performing students were characterized by more unclear FR answers, rather than more incorrect ideas. Finally, we found that FR answers more often represented partial understandings with correct but incomplete ideas, rather than mixed understandings with correct and incorrect ideas. Our results provide important insights that instructors should consider as they select and interpret different question formats. In particular, we will discuss how MTF and FR answer patterns reflect fundamental tensions in question structure and targeting. While MTF questions have an ability to probe specific conceptions, student answers to MTF questions do not allow an instructor to definitively diagnose their underlying thought processes. Conversely, FR questions enable students to describe their understandings in their own words, but rely strongly on the question prompt to cue students to address the targeted conceptions and may fail to detect the presence of incorrect ideas.

Comparing Human and Machine Learning Assessment of Student Reasoning about Natural SelectionMichael Wiser*, BEACON Center for the Study of Evolution in Action; Robert Pennock, BEACON Center; Jim Smith, Michigan State University; Louise Mead, Michigan State University[abstract # 157]Written responses can provide a wealth of data in understanding student reasoning on a topic. Yet they are time- and labor-intensive to score, requiring many instructors to forego them except as limited parts of summative assessments at the end of a unit or course. Recent developments in Machine Learning (ML) have produced computational methods of scoring written responses for the presence or absence of specific concepts. Here, we compare the scores from one particular ML program -- EvoGrader -- to human scoring of responses to a set of questions about natural selection, across a sample of nine courses from seven institutions. Some of the questions were in the set the program was trained on; others are structurally- and content-similar. We demonstrate that the program’s lack of contextual awareness results in its giving credit to students who do not answer the question asked. We find that there is substantial inter-rater reliability between the human and ML scoring, though consistently lower than between the human raters (0.57-0.62 vs 0.69-0.74). Consistent with expectations, inter-rater reliability is higher for questions with which the program was trained than ones on which it was not (0.59-0.63 vs 0.51-0.6), and is better on Key Concepts than Naïve Ideas (as identified in Moharreri, Ha, and Nehm 2014) (0.59-0.65 vs 0.49-0.59). Somewhat surprisingly, inter-rater reliability is higher on questions that involve the loss of a trait than on those involving the gain of a trait (0.62-0.69 vs 0.50-0.58). Sufficient systematic differences remain between the human and ML scoring that we advise only using the ML scoring for formative, rather than summative, assessment of student reasoning.

Beyond the model: Assessing systems thinking in undergraduate biologyJennifer Momsen*, North Dakota State University; Bethany Munson, Bethel University; Madelyn Esperum, North Dakota State University; Sara Wyse, Bethel University[abstract # 23]Current undergraduate education reform identifies complex systems as a core concept of biological literacy. However, the skills and content knowledge comprising systems thinking are largely undefined. We believe two essential components of systems thinking are the ability to delineate the system of interest and to reason

4

4

SABER 2017 Abstracts- Short Talks

dynamically and describe how the system changes over time and in response to a perturbation. Using a model-based pedagogy, we explore students’ systems thinking abilities within the core concept of matter movement and transformation (e.g., carbon cycling). Before and after relevant instruction, introductory biology students constructed concept models of carbon movement in an ecosystem. Students were then asked to use their model to predict how a perturbation to that system would impact carbon cycling. We coded student models for the presence and correctness of key carbon pools and fluxes as a measure of their ability to delineate the focal system. Students’ extended responses were coded to understand whether and to what extent students were able to reason dynamically about an ecosystem. Prior to instruction, introductory biology students struggled to completely and correctly delineate the focal ecosystem. For example, less that 2% of students include decomposers in their carbon cycle models and only 4% of students included plant respiration. Following instruction, student models improved substantially to more completely and correctly delineated the ecosystem: over 75% including decomposers and over 60% correctly including plant respiration. Further, prior to instruction, student reasoning about system perturbations focused mostly on visible pools (e.g., 76% mentioned primary consumers), and only discussed changes to pool size for primary consumers (35%). Flux discussions included consumption (26%) and heterotrophic respiration by visible system pools (5%). After instruction, students’ reasoning about ecosystem perturbations included significantly more pools (df=4, p<0.0001) and fluxes (df=6, p<0.0001), with gains made in discussing plants (photosynthesis & respiration) and decomposers. Overall, we found student understanding of non-visible components of ecosystems was initially limited but improved following instruction. Further, responses to the ecosystem perturbation prompt revealed students exploring multiple ways that systems can respond, evidencing their ability to reason dynamically. Together, these results provide initial evidence of systems thinking in undergraduate biology.

Testing the Test: Are Exams Measuring Understanding?Brian Sato*, UC Irvine; Cynthia Hill, Tufts University; Stanley Lo, University of California San Diego[abstract # 53]Grades in STEM courses are distributed under the assumption that high-performing students have a strong understanding of the course material. Similarly, multiple examples in the STEM education literature present exam performance as equivalent to student understanding. Despite these assumptions, we have little knowledge of student thinking that accompanies high or low test scores. To investigate this relationship, we performed a series of written and verbal exercises with undergraduates studying biology. Twenty-two participants were presented with previously-utilized exam questions and were instructed to include their train of thought, in writing, as they approached each question in addition to providing an exam-like response. Half the participants then participated in a retrospective interview to describe how they arrived at their answer. We coded the exam-like responses, using an instructor-generated rubric, to award an exam-like score. Using this score as a baseline, we then coded the entirety of participants’ writing for their understanding. We found that for 27% of rubric items, there was a discrepancy between performance and understanding. We also identified that retrospective interviews allowed us to gain a greater understanding of participant understanding relative to their written work and helped participants to better articulate their understanding. These results highlight a potential need to re-evaluate our course assessments and to question the understanding those assessments value. Additionally, our work highlights the use of exams as a means to encourage students to engage with science as an ongoing process rather than a process with an end point.

Self or Peer Grading and Reflection on Practice Exams: Which is Better?Mallory Jackson, University of Washington; Alina Tran, University of Washington; Sarah Farrell, University of Washington; Osman Salahuddin, University of Washington; Mary Pat Wenderoth, University of Washington; Jennifer Doherty*, University of Washington[abstract # 149]Practice exams (PEs) provide students with the opportunity for deliberate practice and reflection for metacognition. PEs have been shown to be beneficial. We investigated if grading and feedback by peer or self results in greater learning. Students in a large (n=550) introductory majors’ biology course took weekly online PEs. After taking the PE, the student graded (using instructor’s rubric) either their own or a peer’s PE (grading) and answered two reflection questions: identify keywords in the question (keywords) and give advice and resources to improve performance (advice). We had no a priori hypothesis whether peer or self-grading would lead to greater

5

5

SABER 2017 Abstracts- Short Talks

academic performance. We did hypothesize that students with more expert-like grading, keyword identification and advice would perform better on exams. We used generalized linear regression models with AICc model selection to investigate 1) if course exam performance is explained by treatment or patterns in the data (grading, keyword, advice) and 2) if data patterns are explained by treatment or student demographics. There was no impact of treatment on exam performance. We highlight interesting results of treatment and demographics on grading, keywords and advice. Grading: One question from each weekly PE was graded by two experts with >90% inter-rater reliability. Only GPA was positively correlated with expert PE grade. PE grading accuracy was calculated by comparing expert to student grade. Students with high GPAs who graded peers awarded lower scores, while students with lower GPAs gave much higher scores. This extreme GPA impact on grading accuracy was not seen for students who self-graded. Key Words: Patterns in keywords identified by students were determined using Text analytics and latent class analysis (LCA). Treatment and demographics did not explain variations in patterns. Students in clusters (from LCA) whose keywords most closely matched experts did not perform differently on exams. Advice: Students’ responses were categorized by two experts into four types of advice and eleven types of resources. Treatment and demographics did not explain any variation in groupings. However, the frequency of suggestions to read a question carefully, and use course learning objectives while studying were positively correlated with exam performance. Self-grading is independent of much GPA bias, is as effective as peer grading for improving exam performance and produces fewer student complaints, therefore we suggest using self-grading. Faculty should also reassess the value of identifying keywords and more strongly encourage students to use learning objectives when studying.

FRIDAY - DiversityGroup experience impacts individual performanceElli Theobald*, University of Washington; Sarah Eddy, University of Texas at Austin; Daniel Grunspan, University of Washington; Ben Wiggins, University of Washington; Alison Crowe, University of Washington[abstract # 7]Active learning in college classes and productive participation in the workforce frequently hinge on small, informal groups. However, group dynamics vary, ranging from equitable collaboration to a single individual who dominates. To explore how group dynamics impact student learning, we determined if students who report working with dominators, students’ comfort level, and if working with a friend helped or hindered content mastery (as quantified by statistical models of post-score as a function of group dynamics controlling for pre-score). We asked these questions of students in a large-enrollment introductory biology course after they participated in two different types of intentionally constructed group activities: one loosely-structured activity in which students worked together for an entire class period (we call this the ‘single-group’ activity), and the other highly-structured activity in which students participated in a ‘jigsaw’ wherein students first independently mastered content, then joined groups to teach their peers the specific part of the assignment they had just mastered. We then investigated whether the type of group work or student demographics predicted the likelihood of declaring working with a dominator, being comfortable in their group, or working with a friend, again using statistical models. We found that students who reported a dominator mastered less content than those who did not report a dominator; specifically students who more strongly agreed that they worked with a dominator were nearly 20% less likely to answer an additional question correct on the 8-question post-test, resulting in a post-score that was an entire letter grade lower than students who did not report working with a dominator. Similarly, when students were comfortable in their group, content mastery increased by nearly 30%. Working with a friend was the single biggest predictor of being comfortable, although working with a friend did not directly impact performance. Students who worked with a friend were 425% (i.e., 5.25 times) more likely to feel more comfortable. Finally, we found that students were nearly 70% less likely to agree that someone dominated their group during the jigsaw activities than during the single group activities. We conclude that working with a friend and group activities that rely on positive interdependence, include turn-taking, and have explicit prompts for students, such as our jigsaw, can help increase content mastery thus help reduce the negative impact of inequitable groups.

Sex role representation in evolution textbook imagesSarah Spaulding*, University of Louisville; Linda Fuselier, University of Louisville Biology Department; Perri Eason, University of Louisville; Kasi Jackson, West Virginia University

6

6

SABER 2017 Abstracts- Short Talks

[abstract # 108]As undergraduates engage in curriculum, especially in upper-level courses, they learn both disciplinary knowledge and disciplinary culture. During this process, students may be viewed as engaged in legitimate peripheral participation in a sociocultural community[1] that influences a student’s acceptance of and into that community. College students are exposed to disciplinary culture through their interactions in the classroom. Textbooks, as one of the most influential mediators of these interactions, act as enculturation devices for future biologists[2]. Textbooks guide curriculum and epistemology by communicating what information is relevant within a discipline[3], and the images presented in textbooks send influential, lasting messages which play an increasingly important role in conveying scientific concepts[4,5]. While images can improve student learning[6], they are often interpreted as reflecting reality without consideration of their sociocultural context[7]. The material covered in evolution textbooks, particularly related to sexual selection, could make possible the reinforcement of gender-stereotypic sex roles. We analyzed images in popular evolution textbooks that present content on sexual selection (SXSL) theory. We asked the following research questions: (R1) Do evolution textbook images demonstrate uptake of change in the field? (R2) Are males and females portrayed as equally important to the understanding of SXSL? (R3) How are the images of the sexes situated in the textbook and to what degree do they reinforce gender stereotypical sex role representations as applied to humans? We examined images within a time series of five textbook editions, as well as the latest textbook editions from four publishers; these books comprise over 96% of the evolution textbook market. Image characteristics were coded by multiple investigators using published criteria[8], and interrater reliability was calculated. Despite recent social and methodological changes in SXSL research, we found that most textbook images portrayed a classical view of SXSL and that there was little attention given to SXSL on females or the expanded sex roles typical of a more realistic, complicated view of SXSL. Images of males were more common than images of females, females were depicted for fewer concepts than males, and images of males and females differed in structure such that stereotypical sex roles were reinforced. This study highlights the need for uptake of expanded roles in SXSL theory, greater female representation in evolution textbook images, and attention to messages about stereotypes that are communicated through visual representation. In the future we will analyze how students interpret these images with relation to their socioeconomic and educational background.

Development of cultural intelligence and communication skills using Ecotonos, a simulation role-playing activityKatie Nemeth*, University of Minnesota Duluth; Amy Prunuske, Medical College of Wisconsin[abstract # 69]Historically a curriculum that addresses cultural diversity has been omitted from the sciences. Cultural diversity exercises can help participants reflect on the unconscious biases they bring to the classroom and make clear that each person contributes unique strengths to the learning space. Teamwork across nationalities and disciplines is essential in today's science environment. For these diverse groups to be successful, they must employ strategies that promote inclusivity and innovation. Due to the abstract nature of inclusivity and diversity, these can be difficult concepts to teach. Simulations are underutilized in science education but can be an effective way to provide experiential training that allows students to learn through participation and reflection. Ecotonos is an activity where students, under factious cultural identities, are asked to actively problem solve in group settings. During the first week of class, undergraduate students (n=242) enrolled in a general biology laboratory course participated in the Ecotonos activity. Participants were asked to complete a Cultural Intelligence (CQ) survey before and after participating in the activity. CQ is a method to assess one’s ability to adapt to new cultural experiences and is composed of four factors: metacognitive, cognitive, motivational and behavioral. Students demonstrated variable gains in each of these areas when analyzed with a paired t-test. Student reflection responses were coded to identify common themes. Students found that the activity was an effective way to become aware of the existence of cultural differences, but that they would benefit from additional experiences that help them to resolve conflict between different groups. Participation in Ecotonos simulation is effective for increasing individual cultural awareness for group work.

Reflection assignments and academic performance in Introductory BiologyBryan Dewsbury*, University of Rhode Island; Marcy Kravec, Florida International University[abstract # 115]

7

7

SABER 2017 Abstracts- Short Talks

A growing body of research has shown that short reflection assignments can have cathartic effects on student performance for the subsequent years of their college experience. These effects were shown to have a disproportionately positive effect on students from disadvantaged backgrounds, suggesting that written reflection can play a key role in mitigating some of the perceived deficits associated with students' historical context. In our study, we gave students two reflection assignments, one at the beginning and the other at the end, to two introductory Biology classes at separate universities. Both institutions were large, public research universities but one of the institutions is designated as a Hispanic Serving Institution (HSI). Students wrote 500-word reflections initially on an essay entitled 'I believe', and ended the semester reflecting using the prompt 'Letter to a Future Freshman (LTFF)'. Using grounded theory, we coded the reflections for common themes. We also dis-aggregated the data to see if their were any patterns between the themes and the demographics of our specific student populations. Students from both institutions reflected on similar theme patterns in the initial reflection, focusing on themes of maturity and mindsets as the core ideals of their life beliefs. In the LTFF assignment, there were overlapping themes of the need to socialize, access campus resources and to manage one's time. Students from the non-HSI institution however spoke much more about the need to contact family, protecting their mental and physical health, and trying to find the right balance between all of their obligations. While many students at the non-HSI reflected on contacting family and socializing, the URM students at this institution reflecting in this way were more likely to earn a productive grade in the introductory biology class. Our results indicate that while reflection assignments matter, the social context around the student can dictate the nature of the reflection. They also suggest that the specific themes that students reflect on, above and beyond the simple act of the reflection, might be extraordinarily predictive of how they perform in the classroom.

Active Learning as a Source of Differential Anxiety for Female and Underrepresented Students in Introductory Biology Courses Margaurete Romero*, University of Tennessee; Ben England, The University of Tennessee; Jennifer Brigati, Maryville College; Beth Schussler, "University of Tennessee, Knoxville"[abstract # 112]Instructors have been encouraged to incorporate active learning (AL) into their classrooms to increase student engagement and metacognition (AAAS, 2011). Evidence has shown that active learning promotes greater student learning compared to traditional lecturing across STEM disciplines (Freeman et al., 2014). However, research in our lab has shown that a subset of students in large introductory biology classes experience anxiety in response to the use of AL. This research examined whether this anxiety was experienced differentially by particular sub-populations of students. Specifically, if anxiety differed between 1) genders (females versus males) or 2) underrepresented ethnic groups (nonwhite versus white). We asked students from three lecture sections of the first semester introductory majors (Organismal Biology) and three sections of the second semester introductory majors (Cellular Biology) courses to rate their levels of four types of anxiety (general, test, social, and communication) and anxiety toward four AL activities (clickers, cold calling, volunteering answers, and group work). Overall, 415 students completed the surveys; 20% of students identified as part of an underrepresented group and 72% of students identified as female. Treating each lecture section separately, we found that females felt significantly more test and communication anxiety than their male classmates in each Organismal Biology section (significance denoted by p<0.05). In two of the three Organismal Biology sections, females also reported more general anxiety, as well as anxiety toward clicker questions, cold calling, and volunteering answers. In one section of Cellular Biology taught by a female professor, females felt more communication anxiety than males; there were no other gender differences in any of the Cellular Biology sections. Underrepresented students had higher anxiety than white students in general anxiety, social anxiety, and clicker use in one section of Organismal Biology and for group work in two sections of Organismal Biology. Underrepresented students only had higher anxiety than white students for cold calling in one section of Cellular Biology. Overall, the patterns of anxiety between genders and underrepresented students were very different among sections, aspects of anxiety, and anxiety-inducing AL type. These data demonstrate that although lecture section teaching practices or classroom climate differentially contribute to student anxiety toward AL, this anxiety is higher in the first semester of introductory biology, especially for female students. Additionally, the differences in anxiety patterns between females and underrepresented students signal challenges for instructors seeking to use AL in ways that maximize learning for all students.

8

8

SABER 2017 Abstracts- Short Talks

SATURDAY – Professional Development IN Graduate SchoolAnnotated Primary Literature: A Professional Development Opportunity for Graduate Students in STEM education and communicationMelissa McCartney*, Florida International University[abstract # 39]Graduate students in STEM fields represent a pool of talent capable of improving both STEM education and outreach. The culture regarding the inclusion of educational pedagogy and communication skills into traditional graduate school programs is beginning to change, and research supporting the benefits of this training is beginning to emerge. One way graduate students can participate in education and outreach is helping to develop content for Science in the Classroom (SitC) (http://scienceintheclassroom.org/), a resource aimed at making scientific research papers more accessible to an undergraduate audience through the addition of annotations. SitC relies on graduate student volunteers to write the annotations, essentially turning the talents of one group (graduate students) into resources for the other (undergraduates). In order for this system to be successful, graduate students must be properly trained to annotate a scientific paper. Annotators completed a mini-course consisting of six training videos covering topics such as an introduction to discipline-based education research, the complexities of academic language, and the science of science communication. Annotators were also required to complete an accompanying pre- and post-course survey, adapted from several previously verified surveys and consisting of both qualitative and quantitative data collection, which measured changes in their views on science education and communication as well as any gains in how they view themselves as a member of the scientific community. Data analysis indicates that participants report a deeper understanding of STEM education research efforts as well as more confidence in their science communication skills and a greater sense of being a member of the scientific community. SitC annotations were also analyzed for readability. Using readable.io, a web-based application that uses several different readability algorithms, including Flesch-Kincaid, Gunning Fog, and the SMOG index, SitC annotations are scored and compared to the original text of the scientific article. Preliminary results indicate that the research articles are written at a grade 22 level, suggesting an audience of graduate students and above, while SitC annotations are written at a grade 13 level, which is appropriate for our target audience of undergraduates. Collectively, this data supports the use of annotation training as a value-added component of graduate training, rather than as a diversion from research time.

Graduate Student Perceptions of Evidence-based Teaching PracticesEmma Goodwin*, Portland State University; Miles Fletcher, Portland State University; ERIN SHORTLIDGE, PORTLAND STATE UNIVERSITY[abstract # 65]Low undergraduate retention rates in life sciences have led to national calls to promote adoption of evidence-based teaching practices in STEM education. Critical to this shift are graduate students, who in their roles as teaching assistants directly impact current undergraduate learning experiences. Further, the training and teaching graduate students undergo inevitably impacts the quality of future undergraduate courses—experiences from graduate teaching assistantships (GTAs) are often the primary teaching experience that incoming faculty have. Therefore, understanding current experiences of graduate students with regard to evidence-based teaching (EBT) practices is necessary to identify our national progress in the adoption of EBT in higher education. Towards this goal, we aimed to explore the following research questions: How do graduate students perceive EBT practices? Do graduate students value training in EBT? Are graduate students satisfied with the support they receive in this domain? To answer these questions, we interviewed 33 graduate students representing 25 universities about their understanding of evidence-based teaching practices, and how their graduate programs support their instructional training. Participants included 31 PhD and 2 Master’s students, 22 of which had been GTAs for three or more academic terms. We iteratively developed a coding rubric and two researchers coded 20% of the interviews with an overall inter-rater reliability score of 83%. Of the interviews coded, 85% of participants demonstrated familiarity with EBT practices. Encouragingly, 31% of participants acknowledged that although traditional practices were successful teaching strategies in their own education, traditional strategies do not necessarily “work” for everyone. Most participants (85%) reported that training in EBT is important, and of these individuals, 45% indicated it is valuable because they want to teach, while 36% believe it a necessary skill to get jobs in academia. Despite this, 77% of participants perceive deficiencies in their graduate programs in instructor training: either lacking in

9

9

SABER 2017 Abstracts- Short Talks

instructional techniques, sufficient opportunities to teach, and/or sufficient opportunities to expand their instructional experiences beyond traditional GTA roles. Our analyses reveal that graduate students are highly aware of EBT practices and are interested in developing EBT skills, but are unsatisfied with their instructional training. These data indicate that more attention must be given to training the next generation of faculty, and will provide tangible suggestions on mitigating current deficiencies. Making earnest improvements in GTA training and providing skills for graduate students to effectively use EBT will fill a much-needed deficit while improving graduate student competencies as they enter the workforce.

Is there evidence for perceived trade-offs between research and teaching for life science graduate students?ERIN SHORTLIDGE*, Portland State Univeristy; Sarah Eddy, Florida International University[abstract # 98]There is a pervasive perceived tension between research and teaching for academic faculty, and this perception trickles down to graduate students. In many training programs, graduate students are not encouraged to focus on teaching. Despite this widely held perception, few empirical studies have actually explored the cost of investing in teaching on research. In this study, we explore whether life science graduate students’ awareness, training in, or use of evidence based teaching (EBT) practices reduces three measures of preparedness for a research career. A national sample of life science graduate students was recruited through student and scientific society listservs, department chairs, faculty, and program coordinators. The final sample was composed of 272 life science Ph.D. students from 114 institutions, 77% of which attend graduate programs at R1 institutions, and their research foci range across 20 sub-disciplines. Participants completed a survey asking them to report their awareness of, training in, and implementation of eight EBT practices. Participants also reported several metrics of research preparedness: the number of papers they had published, how adequately prepared they felt for a research career, and their ability to communicate science in a professional setting. General linear models (GLM) were used to model how graduate student awareness of, training in, and use of EBT methods impacted these metrics. We found that there was no relationship between awareness, training, and use of EBT practices, and number of research publications (all p≤0.2), indicating no evidence for a tradeoff between teaching and research productivity. In addition, we found a small increase in student’s reported adequacy of research training with increasing awareness, training, and use of EBT practices (all p≤0.001). Lastly, with increasing awareness, training, and use of EBT, there was a trend for students to be more likely to report confidence in communicating their science (all p≤0.08). These data suggest that there is not an obvious trade-off between investment in research and teaching. In fact, some measures of research ability slightly improved with graduate students’ greater investment in EBT. Given that today’s graduate students report engagement with evidence-based teaching and simultaneously progress towards achieving metrics of a career in academia - future science faculty may have the capacity to excel in both research and teaching.

Building a model of how life science doctoral training can influence professional identity as a college teacherAmanda Lane*, University of Georgia; Carlton Hardison, University of Georgia; Ariana Simon, University of Georgia; Tessa Andrews, University of Georgia[abstract # 44]Despite repeated calls for reform in undergraduate biology education, many courses are still taught traditionally. One potential explanation for this is that many scientists have professional identities that do not include teaching. Professional identity encompasses how a person sees themselves as a professional, including how they see their responsibilities and contributions. A teaching identity is one potential component of professional identity for college faculty. We aimed to understand what contributes to and hinders teaching identity by investigating life sciences doctoral students. This is a novel and critical step toward addressing possible tensions between professional identity and instructional change in undergraduate biology education. Our work was informed by a theoretical framework that posits four domains that contribute to an identity. We explain these in terms of a teaching identity: interest (desire to teach), performance (perception of ability to perform teaching tasks), competence (perception of ability to understand teaching), and recognition (being recognized as a teacher). We performed semi-structured interviews with 29 students in years 2-7 of a PhD in a life science discipline. Questions addressed career plans, how participants see themselves as professionals, how their advisors view teaching, and whether participants perceive tensions between research and teaching in their school environment. We conducted iterative and systematic qualitative content analysis of interview transcripts to generate a hypothetical model of

10

10

SABER 2017 Abstracts- Short Talks

the factors affecting teaching identity. As predicted by our guiding theoretical framework, interest in teaching was required to develop a teaching identity. Interest arose before graduate school or was sparked by graduate teaching experiences. Additionally, interest was a key driver for students to pursue teaching professional development (e.g., classes, teaching certificate) and teaching experiences. From these experiences participants reported gaining knowledge, skills, the ability to “talk the talk” (competence), and increased comfort as a teacher (performance). Feeling ownership in a teaching experience contributed considerably to teaching identity via all four domains. Research advisors influenced participants’ perceptions of whether they had access to teaching-related opportunities and the value of those opportunities. In this way, advisors indirectly contributed to or hindered teaching identity. Our model proposes important considerations for graduate students, research advisors, and graduate training programs. For example, students benefited from early teaching opportunities, informal contracts with advisors about balancing research and teaching, and informed feedback on their teaching. The new model generated by this work can guide reform of doctoral training at one institution, and future investigations will expand its generalizability.

The PhD Panic: Examining the factors and impacts of teaching anxiety in biology graduate studentsMiranda Chen*, University of Tennessee, Knoxville; Beth Schussler, "University of Tennessee, Knoxville"[abstract # 94]Graduate students in the United States have been suffering from increased anxiety over the last several decades, affecting not only the overall mental health of graduate students, but also reducing student retention in graduate programs (Kinman, 2001; Panger et al., 2014). Teachers with high teaching anxiety can negatively impact student learning (Marso and Pigge 1998; Roach, 2003), yet the impacts of graduate student teaching anxiety and its effect on student learning is not well studied. Biology Graduate Teaching Assistants (GTAs) teach over 91% of freshman Biology labs and discussions nationally, thus GTA teaching anxiety could have broad influences on the quality of undergraduate education. In this yearlong longitudinal study, we investigated Biology GTA teaching anxiety at a large research-intensive southeastern university. This study used validated surveys and interviews to: (1) capture the current state of teaching anxiety of Biology GTAs; (2) determine the causes and consequences of this anxiety; and (3) track changes in anxiety levels over a full academic year. Ninety-one graduate students across four Biology departments responded to this survey and 24 were subsequently interviewed to collect in-depth perceptions of teaching anxiety. Using multiple linear regressions, we found that self-efficacy and teaching experience significantly predicted GTA teaching anxiety (R2= 0.54, p <0.001). There were no significant differences in reported teaching anxiety between gender or among departments; however, perception of departmental support for teaching differed among students of the four Biology departments (F(4, 86) = 6.15, p = 0.02). Preparation for teaching also significantly correlated to teaching self-efficacy (r = 0.22, p = 0.04). Interviews revealed that GTAs with low teaching anxiety fell into two categories: (1) GTAs with high teaching self-efficacy who envisioned teaching as part of their future career; and (2) GTAs who were indifferent to teaching and perceived it only as a job to fund their graduate education. Thus, GTA professional identity may be an important variable to consider when measuring teaching anxiety. This longitudinal study is continuing for a second year to provide insight into how self-efficacy, preparation, and identity of GTAs impact teaching anxiety. Results from this study will inform future teaching professional development activities for GTAs, and encourage greater awareness and dialogue about mental health issues in academia.

SATURDAY- EvolutionDoes evolution acceptance differ across biological scales? A Rasch analysis of the Inventory of Student Evolution Acceptance (I-SEA)Gena Sbeglia*, Stony Brook University; Ross Nehm, Stony Brook University[abstract # 86]Evolutionary theory is central to biological literacy but has been shown to be widely misunderstood. One reason is that acceptance of evolution may play a role in learning about evolution. Consequently, biology educators have attempted to define the construct of evolution acceptance and empirically measure it using survey instruments such as the MATE (Rutledge and Warden 1999). The MATE was criticized by Nadelson and Southerland (2012) for not separating the measurement of student acceptance of microevolution, macroevolution, and human evolution. Consequently, they developed a 24-item, likert-scale instrument known as the Inventory of Student Evolution

11

11

SABER 2017 Abstracts- Short Talks

Acceptance (I-SEA), which assesses acceptance of evolution on three subscales: microevolution, macroevolution, and human evolution. Although a large body of psychometric evidence was used to support validity inferences for the I-SEA, it relied on parametric tests of raw data; no tests confirmed that the Likert items were linear or were expressed on an equal-interval scale prior to psychometric analyses. We performed a Rasch analysis of a large sample of pre- and post-test I-SEA scores generated from three semesters of a large (> 200 student) introductory biology class in order to reexamine psychometric validity and test whether scores support prior claims that student acceptance differs across biological subscales. Using Rasch corrected scores, we report that there was sufficient evidence to treat the ISEA as a multi-dimensional construct. Because Rasch analysis assumes uni-dimensionality, we ran separate Rasch analyses for each subscale to identify if multidimensionality was explained by variation among subscales. Overall Rasch person scores increased significantly but marginally from the pre- to the post-test (= 0.32, df = 5791, p < 0.001). However, despite the expectations of the I-SEA, human evolution and macroevolution person scores did not differ significantly from each other. Conversely, microevolution person scores were significantly different from the other subscales, declining marginally from the pre- to the post- test as the macroevolution and human evolution person scores increased (micro:time = -0.67, df = 5777, p < 0.001). Therefore, the change patterns of the microevolution subscale were distinct from the human and macroevolution subscales, providing some support for dimensionality of student understanding. However, the expectation that the human evolution subscale would be the most challenging and the microevolution subscale would be the least challenging was not supported. Although these findings were replicated with large student samples at one institution, they should be examined in additional instructional contexts with a broader range of participants.

The Impact of Changes in Factors on Evolution Acceptance During a Year of Introductory Biology InstructionRyan Dunk*, Syracuse University; Jason Wiles, Syracuse University[abstract # 47]An understanding of evolution is fundamental to the ability to conceptualize and reason correctly regarding a huge diversity of biological phenomena. Unfortunately, understanding of evolution seems to be tied in a complex relationship to acceptance of evolution, which itself is affected by a host of factors. Beyond evolutionary knowledge itself, these factors range from those closely related to education (knowledge of the nature of science) to those more closely tied to sociodemographic and psychological aspects of personal thought (openness to experience and religiosity). Previous work of ours has shown that in undergraduate students, a knowledge of the nature of science, religiosity, and openness to experience all have a stronger independent effect on acceptance of evolution than knowledge of evolution. In this work, we seek to extend our previous model by investigating the changing nature of the relationship between the factors mentioned above. Our primary concern in this is to gain confidence that the factors previously mentioned influence and change along with acceptance of evolution. While this will not directly confirm a causative relationship, a longitudinal model with factors changing in concert will allow us to make more specific claims about how to effect change in evolution acceptance than our previous cross-sectional models of evolution acceptance. Specifically, we have measured introductory biology students’ acceptance of evolution, knowledge of evolution, openness to experience, religiosity, knowledge of the nature of science, and other variables at four time points throughout a year of biology instruction. The change in these variables was assessed using normalized change and analyzed via regressions to determine correlations between the change in independent variables and the change in acceptance of evolution. In addition, multivariate models of the factors associated with acceptance of evolution were made at the beginning and end of a year of general biology instruction. These models were compared to determine what changes, if any, occurred between the two time points in relative importance and impact of the independent variables on acceptance of evolution.

Investigating Evolution Acceptance in Undergraduate Health Sciences StudentsKelsey Metzger*, University of Minnesota Rochester; Darian Montplaisir, University of Minnesota Rochester; Dave Haines, University of Minnesota Rochester[abstract # 32]Evolution acceptance has been studied in a wide range of student populations including undergraduate science majors, non-science majors, and high school students (Romine et al., 2017). University of Minnesota Rochester offers an undergraduate degree in health sciences (BSHS), which prepares graduates for a variety of health-related career pathways. The curriculum includes a one-semester introductory biology course in which topics in evolutionary biology (e.g. origin and history of life on earth, evidence for evolution, phylogenetic relationships,

12

12

SABER 2017 Abstracts- Short Talks

population genetics, patterns resulting from selection pressures, and speciation) are included. The course is taught using a flipped approach, and with student-centered active learning strategies in an ALC. In this study, we seek to compare evolution acceptance in our population of health sciences students with acceptance of evolution in other populations, and to examine whether or not a change in student responses was elicited after a semester of instruction. In a review of the literature, we identified the Measure of Acceptance of the Theory of Evolution (MATE) as a widely used validated and reliable instrument that would provide a means of comparing outcomes in our population to those that have been reported using the MATE. Pre-test data was gathered on via our online course management system. Students completed the assessment outside of class time and were awarded completion points for submitting responses. Pre-test MATE data indicate that our first year health sciences students’ responses result in an average acceptance of evolution score of 75.7% (N=119), which, according to Rutledge and Sadler (2007), indicates “Moderate Acceptance” (scores 65-76) and is approaching “High Acceptance” (77-88). This initial evolution acceptance score is consistent with scores reported in other populations of first year introductory biology students who are science majors (Rissler et al., 2014). Post-test data is forthcoming, and paired t-tests will be used to compare the pre and post MATE scores. Multiple regression modeling will be used to determine the relationship between MATE scores and measures of academic performances, and between MATE scores and demographic variables such as gender or ethnicity. Following this preliminary assessment of evolutionary acceptance in our undergraduate health science students and change over one semester of instruction in biology, we plan to use the MATE and/or the more recently published GAENE (Smith et al., 2016) instruments to evaluate the effectiveness of curricular interventions in future semesters for their impact on student acceptance of evolutionary theory.

Towards more inclusive evolution education: a call to use cultural competence when teaching evolutionElizabeth Barnes*, Arizona State University; Sara Brownell, Arizona State University[abstract # 14]While evolution is a core concept of biology, many students are uncomfortable learning evolution due to their religious beliefs. Many evidence-based practices exist for reducing perceived conflict between religion and evolution and increasing acceptance of evolution among college students. However, college level evolution instructors may struggle with addressing the perceived conflict between evolution and religion in an effective way because their religious cultural backgrounds and beliefs are often different from their students’ cultural backgrounds and beliefs. In this talk, I will describe three studies that illustrate how the differences between instructors’ and students’ religious cultural backgrounds and beliefs may be contributing to ineffective evolution education. First, we interviewed 32 instructors who were teaching evolution at universities and community colleges in Arizona about their instructional practices. Using grounded theory and content analysis of the interview transcripts, we found that most of these instructors did not use evidence-based practices for increasing student acceptance of evolution and that they cited their own lack of religious beliefs, lack of awareness of their students’ beliefs, and their own negative stereotypes towards religion as barriers. Next, we interviewed 28 Judeo-Christian students taking biology classes at a R1 university in Arizona about their experiences learning evolution. In these interviews, we found that students perceived that their biology instructors had negative attitudes towards religion. These students thought that instructors’ negative attitudes towards religion were not only a barrier to their learning of evolution, but was also a potential barrier to them pursuing a career in biology. Finally, to explore evolution education among instructors who share similar religious cultures/beliefs as their students, we interviewed 32 biology instructors at Christian universities nationwide who were teaching evolution. We found that almost all of these instructors were using evidence-based practices for increasing students’ acceptance of evolution and they often cited their own religious cultural background and beliefs as to why. Although future work is needed to confirm our qualitative findings, these studies suggest a need for cultural competence among college evolution educators who have different religious cultures and beliefs than their religious students. Cultural competence is the ability of individuals from one culture to relate to and effectively communicate with individuals from a different culture. To meet this need, I will conclude the talk with a new proposed framework for evolution education that incorporates cultural competence, Religious Cultural Competence in Evolution Education (ReCCEE).

13

13

SABER 2017 Abstracts- Short Talks

What do we really know about teaching and learning evolution? Using pedagogical content knowledge to inform the future of evolution education researchMichelle Ziadie*, University of Georgia; Tessa Andrews, University of Georgia[abstract # 106]Teacher knowledge has the potential to influence student learning in undergraduate biology. One type of teacher knowledge, pedagogical content knowledge (PCK), has been a fruitful theoretical construct in improving teaching and learning in K12, but has not been used in undergraduate biology. PCK is distinct from both content knowledge and pedagogical knowledge. It is topic-specific and consists of at least three dimensions: knowledge of student understanding, including common difficulties and misconceptions about a topic; knowledge of assessment, including how to evaluate student thinking about a topic; and knowledge of instructional strategies, including problems, cases, and activities that facilitate learning a topic. PCK is critical for effective student-centered instruction because it allows teachers to anticipate difficulties students are likely to have and to plan and implement instruction to address those difficulties. One important source of PCK is research on teaching and learning. Therefore, we systematically analyzed the peer-reviewed literature to identify existing knowledge for undergraduate evolution instruction, and to identify gaps in the knowledge available to teachers. We conducted an exhaustive search for relevant articles in disciplinary journals, ERIC, and Google Scholar using terms related to: educational level, evolution topic, and teaching and learning. Multiple researchers independently analyzed each abstract to determine the PCK dimension(s) addressed, evolution topic(s), and article type (research, essay, review, etc.). We discussed any disagreements until reaching consensus and read papers as needed. We discovered 764 relevant papers. Omitting those published prior to 1990 and those primarily addressing acceptance and beliefs, we further analyzed 424 articles. Only 75 focused on student thinking, 63 of which were empirical investigations. Most of these papers (88%) addressed evolution broadly, natural selection, or phylogenetics. No other topic has been the focus of more than three studies of student thinking. Papers about assessment were rare (n=24), but largely empirical. Many papers (n=335) present instructional strategies. In contrast to our findings regarding research on student thinking and assessment, the vast majority of these papers (78%) were not empirical or relied on self-report. While non-empirical work can provide ideas and inspiration for teachers, it does not contribute to our evidence base to improve evolution instruction. Using PCK as a framework, we have revealed large gaps in existing research on undergraduate evolution education that future studies can address. Further, PCK provides a useful lens for considering the value of future work in evolution education for college instructors and the research community.

SATURDAY – CUREs/ProcessA department-wide curricular reform effort to increase authentic research opportunities shows positive impacts for introductory biology studentsKelly McDonald*, California State University, Sacramento; Allison Martin, California State University, Sacramento; Adam Rechs, California State University, Sacramento; Cody Watters, California State University, Sacramento; Thomas Landerholm, California State University, Sacramento[abstract # 3]The Sustainable Interdisciplinary Research to Inspire Undergraduate Success (SIRIUS) project aims to provide all Sacramento State biology majors with repeated opportunities to conduct authentic research on an impacted waterway that runs through campus. This project arose from an observed disparity between students desiring undergraduate research experiences (UREs) (91% of 295 surveyed) and the number of opportunities available (3% accommodated in faculty labs). Furthermore, an analysis of four semesters of enrollment data revealed that UREs were disproportionately offered to students who were white, male, native freshmen, of higher income, with higher GPAs and with a parent who attended college. Transfer students were less likely to participate in UREs irrespective of GPA. Our solution involves incorporating research experiences, based on the critical elements of Course-based Undergraduate Research Experiences (CUREs), into 12 existing laboratory courses, coordinated around three scientific threads that address human impacts on the American River Ecosystem. Here we describe the impacts of the curricular changes on students enrolled in BIO1: Introduction to Biodiversity, Evolution and Ecology and BIO2: Introduction to Cells, Molecules and Genes. Surveys and focus groups were employed to assess student outcomes related to self-efficacy, science identity and future academic and career plans. We used the Laboratory Course Assessment Survey (LCAS) to evaluate student recognition of critical CURE design elements in the curricula.

14

14

SABER 2017 Abstracts- Short Talks

Significant gains in laboratory self-efficacy were observed for BIO1 and BIO2 students who reported the lowest levels of confidence at the beginning of the semester, and disparities that existed for BIO1 students based on gender and Pell Grant qualification at the beginning of the semester were eliminated by the end of the course. With regard to science identity, 81% and 92% of BIO2 students in two semesters of the CURE felt they were performing “real science,” compared to only 50% in the same course before CURE implementation. Students from the CUREs commonly cited the collection of novel data and the relevance of their projects as explanations. While students from the BIO2 CURE reported a greater interest in obtaining a Ph.D. in science (19% strongly agreed) compared to those before CURE implementation (6%), many students in BIO1 and BIO2 remained undecided about their future academic and career plans after both courses. LCAS results indicated that students in both the BIO1 and BIO2 CUREs recognized curricular elements related to collaboration, discovery/relevance and iteration, and focus group transcripts provided rich data to further explain survey findings.

Characterization of Students’ Experimental Design Approaches in Traditional Laboratories versus Course-based Undergraduate Research ExperiencesDavid Esparza, University of Texas at El Paso; Haidar Ahmed, The University of Texas at El Paso; Jeffrey Olimpo*, The University of Texas at El Paso[abstract # 72]Course-based undergraduate research experiences (CUREs) have been identified as a promising vehicle to broaden students’ participation in authentic scientific opportunities. While recent studies in the bioeducation literature have capitalized upon the mediating influence of CUREs on cognitive and non-cognitive student outcomes (e.g., attitudes and motivation, science process skills development, ability to “think like a scientist”), it remains unclear whether undergraduate students exhibit explicit preferences when designing scientific experiments, and, if so, for what reasons. In addition, the extent to which experimental design preferences are similar or dissimilar between students enrolled in traditional laboratory coursework vs. CUREs requires examination. To address these concerns, we administered the Expanded Experimental Design Ability Tool (E-EDAT) in pre-/post-semester format to students enrolled in either the traditional (n = 60) or CURE (n = 47) sections of an introductory cell and molecular biology laboratory course in the Spring 2016 term. Type of experimental design employed (e.g., repeated measures; treatment vs. control) was assessed using a rubric developed in-house expressly for that purpose. A subset of individuals from both the CURE and non-CURE cohorts (n = 25) was likewise invited to take part in a brief, semi-structured interview at the end of the term, the intent of which was to provide a nuanced account of their experimental design choices and the potential “evolution” of these choices over time. Chi-square analyses demonstrated that students in both cohorts exhibited a preference for the “control vs. treatment” design at the start of the term (p < 0.005 for all analyses); furthermore, this remained the predominant approach used by individuals in both conditions at the end of the course. Descriptive interpretive analyses of interview data reveal that this is likely due to how the scientific method had been conceptualized in students’ prior coursework or in the media, lack of knowledge regarding alternate design strategies, and personal experience. Collectively, these outcomes suggest that an explicit pedagogical focus on how students choose to design experiments and the use and value of various design-based approaches is warranted.

Diving into qualitative data from a CURE on seafood mislabeling to understand how course design influences student outcomes Lisa Corwin*, University of Colorado Boulder; Logan Gin, University of North Carolina, Chapel Hill; Blaire Steinwand, University of North Carolina, Chapel Hill; John Bruno, University of North Carolina, Chapel Hill[abstract # 75]The next generation of scientists and citizens will “face more daunting challenges” (Vision and Change, 2009) and thus need to “develop the temperament necessary for research work” (Seymour et al., 2003). Involvement in research helps students develop the tolerance for obstacles, failure, and ambiguity, temperaments necessary for this next generation of scientists and complex thinkers (Seymour et al., 2003, Laursen et al., 2010. Course-based undergraduate research experiences involve students in research at scale. These experiences involve students in research that is relevant to a community outside of the classroom and provide students with opportunities to make new discoveries, and importantly, to iterate when encountering challenges in order to make scientific progress. Iteration is predicted to increase students’ tolerance for obstacles in science and their ability to work productively to solve challenges. In this study, we investigated this prediction by comparing student experiences in

15

15

SABER 2017 Abstracts- Short Talks

two offerings of a CURE: one in which students faced many technical challenges and thus iterated more within their work, and one where they faced fewer challenges and underwent fewer iterations. We characterized student outcomes in each course to understand how these differences in iteration influenced students’ outcomes, and specifically, their tolerance for scientific obstacles. The CURE involved students in investigations of seafood mislabeling and engaged ~20 students in research for three hours a week. In the first-offering, students encountered numerous technical challenges that increased the iterations necessary to make progress. Conversely in the third iteration, students encountered fewer technical challenges and made progress with fewer iterations. For both offerings, we asked students to complete a post-course survey of five open-ended questions requesting that they characterize their experience and describe challenges they encountered. We also conducted focus groups with two groups of students after they completed the course. We used open-coding with an a-priori set of codes characterizing course design and potential student outcomes to code student responses and focus group transcripts. Themes derived from this data suggest that technical challenges experienced by students resulted in high levels of iteration, and that high levels of iteration led students to develop tolerance for obstacles and shift their view of failure.

Computational UREs versus Wet Bench UREs: Design Features and Comparison of Student Experiences Anita Schuchardt*, University of Minnesota; Catherine Kirkpatrick, University of Minnesota; Daniel Baltz, University of Minnesota; Sehoya Cotner, University of Minnesota; Robin Wright, University of Minnesota[abstract # 187]Identification of positive outcomes associated with different course-based research experiences is necessary to generate guidelines for developing effective curricula. Because of the increased interest in, and demand for, computational skills in biology research, our team has recently implemented in our majors introductory-biology sequence an in silico investigation engaging students in exploration of human microbiome ecology. Database-driven investigations can be cost-effective, and harness the hypothesis-testing power of large data sets while exposing students to different types of computational analyses. However, it is not known whether students will react in the same way to an experience that is computer-based as opposed to one that involves bench work. We discuss the implementation of a student-choice semester-long CURE in three contexts. Specifically, students were given the opportunity to work on computational research (analysis of human microbiome data) or one of two wet-bench experiences involving either a vertebrate organism (zebrafish) or a bacterium. All three options met the definition of undergraduate research experiences (URE), involving collaborative, mentor-guided research on relevant problems that emphasized participation in scientific practices. Over 400 students from two semesters completed a survey at the end of the semester on their interest level, their sense of ownership of the project, and their overall sense of satisfaction with the lab using Likert Scale metrics (Strongly Disagree to Strongly Agree). Regression analysis revealed that the primary determinants of satisfaction were students’ sense of ownership and their interest in the project. Getting their first choice or the URE context were not significant predictors of satisfaction. Moreover, students in the computational lab reported greater sense of ownership than either wet lab experience. Interest in the research experience for the computational group was comparable to the wet lab group that reported the highest interest level. We will discuss design elements, identified from student comments, that may contribute to project ownership, interest, and satisfaction. Our results suggest that the hands-on experiences of wet labs are not necessarily conducive to more positive experiences than are computational experiences. In fact, our findings suggest that in situations where wet lab UREs are not structurally feasible, computational UREs may be a desirable alternative.

Model-Based Inquiry in an Introductory Biology Laboratory CourseSusan Hester*, University of Arizona; Emily Dykstra, University of Arizona; Lisa Elfring, University of Arizona; Jennifer Katcher, Pima Community College; Vesna Pepic, University of Arizona; Michele Nadler, University of Arizona; Cheyne White, University of Arizona; Molly Bolger, University of Arizona[abstract # 136]Introductory biology laboratory courses provide a unique teaching and learning opportunity, as they provide the first glimpse of “doing science” to many students. The types of activities in which students engage in these lab courses, however, are often misaligned with the ways of thinking that take place in authentic research settings and may give students a skewed vision of the processes of science. Designing curricula that fulfill the goal of engaging students in authentic scientific practices is challenging, particularly at institutions such as large public universities

16

16

SABER 2017 Abstracts- Short Talks

where resources per student are particularly limited and institutions such as two-year colleges where students and faculty may feel disconnected from the scientific research community. Model-based inquiry (MBI) is a pedagogical approach that engages students in development, testing and revision of explanatory models as a centerpiece of authentic scientific practice. This approach has proven successful in settings as diverse as K-12 classrooms and undergraduate physics and chemistry classrooms, but has not previously been implemented in an undergraduate biology laboratory. Our team is building an MBI-based curriculum for an Introductory Molecular and Cellular Biology laboratory course offered at both a large public university and a two-year college. In Spring 2017, we piloted MBI units at a large public university. Students collaboratively devised and tested their own models to explain phenomena including membrane transport, differential thriving and interdependence of bacterial species in different environments, and phototaxis of unicellular eukaryotes. In Fall 2017, we will pilot these units at a two-year college and a fully revised MBI curriculum at the large public university. To document student learning, we collected audio of student conversations in lab and student written work. To measure impact on student autonomy and attitudes towards science, we administered surveys (Project Ownership (Hanauer and Dolan 2014) and CLASS-Bio (Semsar et al. 2011)), and conducted interviews about the nature of science (extended from Russell and Weaver 2010) with students in both the pilot and unrevised sections at the large public university. We will present examples of how we applied MBI ideas to design instruction in this context, and share excerpts from student dialogue and written work to illustrate how students are reasoning about models and experiments in the revised lab course. We will also discuss our preliminary results about the impact of the revised curriculum on student attitudes and perceptions of autonomy. This work is supported by a grant from the National Science Foundation (DUE-1625015). Key words: laboratory instruction, models and modeling, science process skills

SATURDAY- Active Learning IIInvestigating active learning in STEM from middle school to university: Examining the impact on educational transitionsKen Akiha*, University of Maine; Michelle Smith, University of Maine; MacKenzie Stetzer, University of Maine; Erin Vinson, University of Maine; Emilie Brigham, University of Maine; Justin Lewin, University of Maine; Brian Couch, University of Nebraska-Lincoln[abstract # 43]Despite the need for a strong Science, Technology, Engineering, and Math (STEM) workforce, there is also a high attrition rate for students who intend to complete undergraduate majors in these disciplines. Currently, students who leave STEM degree programs often cite uninspiring instruction in introductory courses, such as traditional lecturing, as a reason. While undergraduate courses play critical roles in STEM retention, little is understood about the instructional transitions students encounter upon moving from secondary to post-secondary STEM courses. This study compares classroom observation data, collected using the Classroom Observation Protocol for Undergraduate STEM (COPUS) (Smith et al., 2013), from over 450 middle school, high school, introductory-level university, and advanced-level university classes across STEM disciplines. To compare instruction between the different levels we used relative abundance, defined as the percentage of the total number of observation codes marked as a specific code or group of similar codes, and relative frequency, defined as the percentage of time intervals marked with a specific code. We find similarities between middle school and high school classroom instruction, which are both characterized by more small-group activities, peer discussion, and instructors guiding students as they are working. In contrast, introductory and advanced university instructors devote significantly more time to more passive teaching strategies, such as lecturing. Furthermore, we detect no differences in the amount of active learning present in introductory and advanced university courses. In this study, middle school, high school, and university instructors were also surveyed about their views of what STEM instructional practices are most common at each educational level and why the instructors hold those perceptions. The percent of each prediction, as indicated by multiple choice answer, was compared between instructors at middle school, high school, and university levels and an open coding process was used to analyze the instructors’ explanations about why they selected a specific answer. Instructors from all levels struggled to predict the level of traditional lecturing practices and often expressed uncertainty about what instruction looks like at levels other than their own. These findings suggest that more opportunities need to be created for instructors across multiple levels of education to share their active learning teaching practices and discuss the transitions students are making between different educational levels.

17

17

SABER 2017 Abstracts- Short Talks

Comparison of traditional and gamified student response systems: does more fun come at a cost?Justin Shaffer*, "University of California, Irvine"; Ethan Luong, University of California, Irvine; Kristen Yabuno, University of California, Irvine[abstract # 133]Student response systems (often called “clickers”) are typically used in college science courses as a form of active learning to increase engagement and promote learning. Recent modifications to student response systems include “gamification” where game-like elements (such as competitions, earning points for speed, winning medals, etc) are added with the goal of further increasing participation, engagement, and excitement in the classroom. However, little is known about the relative impacts of traditional student response systems compared to those of gamified systems when both are used in the same course. The goal of this study was to compare student performance with and perceptions of a traditional student response system (iClicker) and a gamified version (Kahoot!) that were both used in the same course. Our hypotheses were that student performance from iClickers would be more predictive of exam performance than Kahoot!, but that students would perceive Kahoot! as being more fun and engaging than iClickers. Students from two sections of a large enrollment human anatomy course (n = 254) answered in-class iClicker questions for participation credit and voluntarily answered in-class Kahoot! questions for no credit. Students also completed end-of-course surveys rating their perceptions of both systems. Multiple linear regression models were used to analyze exam performance as a function of student performance on iClicker questions and Kahoot! questions (while controlling for student demographics). The models showed that both iClicker performance and Kahoot! performance were significantly positively correlated with exam performance with nearly identical effect sizes. Additionally, student survey data showed that over 80% of students rated both systems as being both fun and effective ways to learn. However, while >95% of students reported that while iClickers should be used in every day of class, approximately two-thirds of students reported that Kahoot! should only be played once a week. These results suggest that student performance with traditional and gamified student response systems may be similarly correlated with exam performance but that gamified systems may need to be used more sparingly than traditional systems in order to maintain high levels of engagement.

When Group Work Doesn’t Work: Insights from Student Interviews and Comments on Peer EvaluationsYunjeong Chang, University of Virginia; Peggy Brickman*, University of Georgia[abstract # 38]Introducing group work in college science classrooms can lead to noticeable gains in student achievement, reasoning ability, and motivation. However, many faculty express concerns over how to best compose groups to maximize student learning and ensure productive interactions. Proponents of group work have recommended social learning supports that include group contracts, role assignment, and anonymous peer evaluations as mechanisms to increase individual accountability and ensure equitable contributions. We implemented these supports and then analyzed how the overall learning outcome for a group impacted student perceptions of group work, and whether students were communicating effectively to their peers using social learning supports. Students from 65 groups provided a total of 1341 comments on both mid- and end-of-semester peer evaluations. Drawing from theoretical models of cooperative learning that focus on motivation and perception (self-determination and social interdependence theories), we coded the peer evaluation comments starting first with priori codes and adding emerging codes as necessary. We found that the anonymous comments students left on peer evaluations differed only subtly between high and low performing groups. Students grouped together with other students of differing ability on tests (heterogeneous) left a larger variety of comments and were more likely to leave comments with a negative perspective (31.25% on midsemester evaluations and 28.57% on final) compared to homogenous groups (0% on mid and 18.18% on final). We found that students in heterogeneous groups who earned lower test scores received higher mean peer evaluation ratings when they were in higher performing groups (M = 101.21, SD = 1.99, SE = .57) than in lower performing groups (M = 86.96, SD = 25.63, SE = 3.78). In-depth interviews with 15 students who scored along a range of abilities on tests from the upper, mid and lower thirds of the class from both high and low performing groups helped verify and support the breadth and overall representation of comments. Students, whether in higher and lower performing groups, noted unequal contributions by their members. However, they all praised the social support provided by groups. Students who scored highly on tests were more likely to recognize the benefits of groups, regardless of their group’s overall performance level, while lower scoring students perceived group work as time consuming “busy work” with little cognitive benefit. Based on the results,

18

18

SABER 2017 Abstracts- Short Talks

we discuss future research and practical suggestions for instructors teaching large enrollment courses who wish to promote equitable collaborative learning.

Active Learning, Anxiety, and Alienation: Potential impacts on student persistence and successBen England*, The University of Tennessee; Jennifer Brigati, Maryville College; Beth Schussler, "University of Tennessee, Knoxville"[abstract # 96]Calls to reform undergraduate teaching in biology (AAAS, 2011) have resulted in increased use of active learning, which has been shown to increase student success and retention in STEM courses (Freeman et al., 2014). However, research in our lab has found that 16% of students in our introductory biology classes experience anxiety in response to active learning. If this anxiety interferes with the benefits of active learning for this subset of students, they may suffer differential and negative persistence or performance compared to other students. Therefore, having a better understanding of how anxiety impacts students over their introductory course sequence is important to support student learning in biology. This project investigated (1) how student anxiety changed over a two semester introductory biology sequence at a large research university and (2) the implications of this anxiety on student persistence and achievement. The potential participants were students enrolled in the introductory biology sequence (six possible sections in the fall and six in the spring) over the 2016-2017 academic year. All students who completed all four anxiety surveys were the final participants (N = 72, to date). Students responded to validated items measuring their levels of general, test, communication, and social anxiety, as well as how anxious they felt when clickers were used in class, when they were called on to answer a question, when they were asked to volunteer to answer a question, and when they were asked to complete group work. Data regarding student final grade, self-efficacy, and major were also collected. Repeated measures ANOVAs revealed that general class anxiety (F=3.440, p<0.05) and communication anxiety (F=12.718, p<0.01) significantly decreased from end of fall semester to beginning of spring semester, and self-efficacy increased from end of fall to beginning of spring semester (F=21.105, p<0.01). General class anxiety levels increased as student final fall semester grade decreased (F=17.399, p<0.01), and students starting (t=-2.618, p < 0.05) and ending (t=-4.599, p<0.01) the fall semester with significantly higher anxiety were less likely to persist in the major. This study revealed an important outcome for educators to consider: although active learning benefits the majority of students, introductory courses aligned with Vision and Change may potentially hinder a vulnerable subset of students who, in a traditional lecture class, may have experienced little anxiety. This study will inform recommendations to promote an inclusive active learning classroom to improve retention and success in introductory biology.

Self-efficacy and control of learning beliefs predict achievement in entry-level biology courseAnupama Shanmuganathan*, Washington & Jefferson College[abstract # 81]The majority of studies within biology education research focus on student cognition while little emphasis is placed on measuring student affect. Yet it is becoming apparent that students’ perceived self-identity of themselves as a biology scholar shapes their learning and achievement. This study explored the effect of student motivation on achievement in an introductory biology course. Based on prior research and social cognitive framework, it was hypothesized that students who are highly motivated and confident in their ability to succeed will report a better course performance. Students enrolled in introductory biology at our college were part of the study. Students’ motivation was measured using the Motivated Strategies for Learning Questionnaire (MSLQ) developed by Pintrich et al. (1991), which is a self-report Likert scale instrument grounded in the social-cognitive view of motivation. The reliability of survey items constituting various sub-scales (intrinsic goal orientation, extrinsic goal orientation, task value, control of learning beliefs, self-efficacy, and test anxiety) was evaluated using Cronbach’s alpha and were found to range from 0.66 – 0.93, which compared well with those of prior studies. A multiple linear regression was employed to ascertain the contribution of each motivational subscale to course achievement. We found that motivation contributed to 36.6% of variation in the course grade: F(6, 92) = 8.270, p<0.001. Of the various motivational constructs, control of learning beliefs (β = 0.54, p < 0.05) and self-efficacy (β = 0.36, p < 0.01) were most predictive of course achievement. To parse out the effect of demographic, educational and experiential factors from the effect of motivation on student achievement, we used a mean split on self-efficacy scores. We then controlled individually for each factor (gender, year of study, race, SAT scores or prior knowledge as measured by a pre-test score). When gender, year of study, race or SAT scores were controlled, the mean course

19

19

SABER 2017 Abstracts- Short Talks

grade for the high self-efficacy group was always significantly greater than that of the low self-efficacy group. When controlled for prior knowledge, interestingly, self-efficacy mattered only when the students had a high prior knowledge of biology. The implications of this study would help educators better understand and harness those predictive motivational factors to improve student achievement and persistence in biology.

SATURDAY- CurriculumLong-term Impacts of Curricular Renovation in Introductory Biology: An 8-Year Case StudyTammy long*, Michigan State University; Jennifer Doherty, University of Washington; Diane Ebert-May, Michigan State University[abstract # 163]Increasingly, biology faculty are incorporating evidence-based pedagogical practices in college courses. Introductory courses, in particular, have been a focus of studies that measure positive impacts in terms of grades, performance on concept tests, attendance, and science attitudes. Fewer studies, however, have explored longer-term impacts. In this study, we report on the process and outcomes associated with an 8-year curricular renovation effort in introductory biology at Michigan State University that began in 2008-09. Bio 2 is the second in a 2-course sequence required for life science majors and focuses on genetics, evolution, and ecology. Our revised course engages students in core disciplinary practices (e.g., modeling, argumentation, data analysis) as a way to learn foundational concepts, uses frequent and rigorous assessments based on real-world cases, and relies on collaborative teams to support learning in and out of class. Shared course materials and weekly voluntary meetings provide interested faculty a source of peer support and feedback. As of 2015, the renovated curriculum had been adopted by 9 of 16 instructors teaching 23 of 53 class sections and 3929 of 8609 students. A Chi-Square analysis of grade distributions reveals statistically higher grades and lower fail rates in the renovated Bio 2 when compared with traditionally-taught sections. However, we observed no overall effect of curricular treatment in analyses of subsequent performance in 6 upper-division courses for which Bio 2 is preparatory or required. Logistic regression analyses were run to evaluate specific impacts on student groups of interest, including first-generation college, students with high financial need, and students underrepresented in STEM. Overall, Bio 2 curriculum (traditional vs. renovated) was not found to be a significant predictor of student performance for any of these groups, although the renovated course was associated with lower performance for first-generation students in a 400-level biochemistry course. GPA was the most significant predictor of performance in all courses tested, and males outperformed females in 4 of the 6 courses (p<.05). Our findings contradict 2 long-held assertions about curricular reform. First, despite a prevailing view about faculty resistance to time on teaching, we found colleagues willing to voluntarily commit to collaborations that develop, test, and share materials that support Bio 2 instruction. Second, our data do not support a frequently cited assertion that reducing content to make room for practice-based instruction will compromise students’ preparation for advanced coursework. Indeed, we observed no statistical difference in upper-level grades linked to introductory curriculum.

Concurrent Classes as an Intervention Strategy in Undergraduate Biology and ChemistryJoel Ledford*, UC Davis; Susan Keen, UC Davis; Nicole Sharpe, UC Davis[abstract # 101]Recent analyses of student performance at UC Davis show that incoming freshmen with academic deficiencies suffer long-term effects including increased time to graduation, greater probability of academic probation, and higher dismissal rates. While the reasons affecting student success or failure vary, we have observed that in many cases students who do not perform as well as they expect lack the study skills, time management, and other behavioral components necessary for success. In this study, we test whether or not student performance improves in primary (parent) courses by providing an optional concurrent class (co-class) focused on content reinforcement, study skills, and counseling support. During the 2016-2017 academic year, we ran 23 co-class sections in the introductory biology and chemistry series at UC Davis with 300 student participants. Enrollment in each co-class section was limited to 15 students and met three hours per week, with two hours devoted to content reinforcement and one hour focused on advisory support. Content reinforcement included review of course learning goals, guided practice of example problems, and simulated exams. Advising support included exercises on study skills, time management, self-care, and topics addressing student outlook towards learning. Co-class sections were led by a combination of faculty instructors, academic advisors, counseling staff, and teaching assistants. In

20

20

SABER 2017 Abstracts- Short Talks

progress evaluation of the effectiveness of the program was driven by a combination of pre- post- and weekly surveys provided to students. Results varied by parent course, but show a maximum gain of 10% in course grade above controls. Analysis of student outlook and behavior showed significant gains across all parent courses.

Conceptual Elements: A Detailed Framework to Support and Assess Student Learning of Biology Core ConceptsJanet Branchaw*, University of Wisconsin - Madison; Tawnya Cary, Beloit College[abstract # 138]The Vision and Change in Undergraduate Biology Education: Call to Action report (AAAS, 2011) has inspired and supported a nation-wide movement to restructure undergraduate biology curricula to address overarching disciplinary concepts and competencies. The report outlines the concepts and competencies generally, but does not provide a detailed framework to guide the development of the learning outcomes, instructional materials, and assessment instruments needed to create a reformed biology curriculum. We have developed a detailed Vision and Change core concept framework that articulates key components that transcend sub-disciplines and scales for each overarching biological concept, the Conceptual Elements (CE) Framework. The CE Framework was developed using a grassroots approach of iterative revision and incorporates feedback from over 60 biologists and undergraduate biology educators from across the United States. The final validation step resulted in strong national consensus with greater than 92% of responders agreeing that each core concept list was ready for use by the biological sciences community, as determined by scientific accuracy and completeness. Details regarding how educators and departments can use the CE Framework to guide and document reformation of individual courses and entire curricula will be included. Efforts to disseminate, test and refine the framework at institutions across the country will be described.

GenBio-MAPS: A programmatic assessment designed to measure student’s conceptual understanding of core biology concepts across a curriculumChristian Wright*, Arizona State University; Brian Couch, University of Nebraska-Lincoln; Alison Crowe, University of Washington; Scott Freeman, University of Washington; Michelle Smith, University of Maine; Jenny Knight, University of Colorado, Boulder; Mindi Summers, University of Maine; Katharine Semsar, University of Colorado, Boulder; Sara Brownell, Arizona State University[abstract # 189]The Vision and Change report provides instructors with a nationally agreed upon framework of core concepts that all undergraduate biology students should master by the time they graduate. Although the establishment of these core concepts was a critical first step in improving the quality of undergraduate biology education, unfortunately there are limited options by which departments can assess how well they are moving students towards mastery of these concepts. To address this need, an NSF-funded, multi-institution collaboration has developed a suite of programmatic assessments called the Bio-MAPS (Biology-Measuring Achievement and Progression in Science) assessments that measure student’s conceptual understanding of core biology concepts outlined in the Vision and Change report and further articulated by the BioCore Guide. These assessments are designed to be implemented at multiple time points during an undergraduate biology career. Thus, these programmatic assessments allow departments to measure changes in students’ conceptual understanding of biology as they progress through a curriculum and can enable departments to make targeted revisions to their curriculum and track changes over time. In this talk, we highlight one of the assessments, the General Biology Bio-MAPS assessment (GenBio-MAPS), which is aimed at assessing student’s conceptual knowledge of general biology. Using an iterative process of question development, including revising questions based on 1) student feedback, 2) expert feedback, and 3) an initial pilot to over 1900 students nationally, we crafted a final version of the assessment. Each of the questions on the final assessment was aligned to the core concepts and biological subdisciplines in the BioCore guide. This final version was then administered to over 7000 students in 152 courses at 20 institutions nationwide, with cross-sectional data being collected from different students at three time points in the curriculum. Using a suite of psychometric analytical techniques, we found that the questions on the assessment covered a wide range of item difficulties and that a majority of the questions met or exceeded the criterion for adequate item discrimination. Furthermore, we found minimal question bias against various demographic groups. In addition to characterizing the psychometric performance of the test and individual questions, we also analyzed student performance data using mixed-effects modeling, finding that there was a progressive improvement in student performance on the

21

21

SABER 2017 Abstracts- Short Talks

assessment as they progressed through a major, although the gains in student performance varied from question to question.

Collaborating Across the University of Maine System to Improve Student Understanding of Energy and Matter Elizabeth Trenckmann*, University of Maine ; Erin Vinson, University of Maine; Karen Pelletreau, University of Maine; Kimberly Borges-Therien, University of Maine; Farahad Dastoor, University of Maine; Jason Johnston, University of Maine; Eric Jones, University of Maine; Peter Nelson, University of Maine; Jenn Page, Hurricane Island; Nancy Prentiss, University of Maine; Judith Roe, University of Maine; Joseph Staples, University of Maine; Emma Toth, University of Maine; Michelle Smith, University of Maine[abstract # 71]Biology students require broad preparation for diverse careers including agriculture, natural resources management, and laboratory research. Concurrent with this need, employers in the state of Maine are seeking applicants who have scientific skills to solve problems related to locally relevant economic systems and develop ways to foster economic growth. University of Maine (UMaine) biology faculty have begun responding to this need by transforming courses to move beyond the memorization of facts in order to increasingly focus on workforce-ready skills, such as problem solving. To support these efforts, this project brought together biology faculty from six different campuses in the UMaine System to collaboratively assess student knowledge of ecology and evolution, share student learning data, develop an economically-relevant instructional activity, and iteratively revise the materials. In total, 715 undergraduate students from multiple UMaine campuses took the EcoEvo-MAPS assessment (Summers et al., submitted) that targets foundational concepts in ecology and evolution. The results revealed that students across the UMaine System struggled to understand the role of nutrients (63% incorrect), carbon dioxide (68% incorrect), and light energy (72% incorrect) in ecological systems. To help students overcome common conceptual difficulties in these areas, the faculty collaborated with an industry partner, the Hurricane Island Center for Science and Leadership, to develop a student-centered in-class activity that focuses on locally relevant industries including forestry, potato farming, and kelp aquaculture. The activity addresses the role of matter and energy in photosynthetic organisms, the relationship between photosynthesis and global carbon dioxide cycles, and the impacts of rising global atmospheric carbon dioxide on economic industries that rely on these processes. The activity was taught in 13 classrooms, and student performance was assessed using a multiple-choice pre/post-test, pre/post Automated Analysis of Constructed Response Questions (http://create4stem.msu.edu/project/aacr/questions), in-class clicker questions with peer discussion, and exam questions. Student learning results and observation data, collected using the Classroom Observation Protocol for Undergraduate STEM (Smith et al., 2013), were used to iteratively revise the instructional materials. Here we report on how the activity improved student learning and promoted problem solving skills, and how combining the expertise of UMaine System faculty and the Hurricane Island Center for Science and Leadership provided the opportunity to integrate biological concepts with economic development. We will also discuss how the chance to collaboratively work together inspired faculty to try new student-centered instructional techniques in a supportive community.

SATURDAY – CURES/ProcessThe relationship between students’ explanations and their interpretation of inquiry investigationsEmily Scott*, Andy Anderson, Kirsten Edwards, Michigan State University[abstract # 135]A Framework for K-12 Science Education identifies (a) constructing explanations, (b) analyzing and interpreting data and (c) engaging in arguments from evidence as three essential practices for students to develop in order to form a cohesive understanding of science. Here, we build off previous work developing a learning progression framework describing the ways students engage in these practices to investigate connections among the three practices above. We used our learning progression framework to code interviews from middle school, high school, undergraduate, and graduate students (n=25) who were asked to explain and critique investigations that involved tracing matter through carbon-transforming processes (animal growth, decomposition or plant growth). We also scored ~440 middle and high school student written assessments similar in structure to the interview protocol to validate trends we found in the interviews. We found a weak correlation between students’ ability to construct explanations with their ability to develop arguments from evidence (Pearson’s r comparing explanation scores with

22

22

SABER 2017 Abstracts- Short Talks

interpreting claims scores = 0.40, t= 9.04, p<0.001; Pearson’s r comparing explanation scores with critiquing claims with evidence = 0.36, t=8.11, p<0.001). These results suggest student proficiency in constructing explanations does not necessarily predict their ability to engage in arguments from evidence. For example, students can be knowledgeable about how systems work without understanding how to use matter-tracing data about that system to support their explanations. These findings have implications for instructional design, especially about the preparation and scaffolding that students need to interpret the results of matter-tracing investigations. Key words: Conceptual understanding and conceptual change, Assessment of student learning and instructional innovation, Classroom assessment

Improving Science Writing in an Introductory Biology Course One Sentence at a Time.Christelle Sabatier*, Santa Clara University; Tracy Ruscetti, Santa Clara University; Katie Krueger, Santa Clara University[abstract # 95]In laboratory courses, instructors commonly assess student understanding through written laboratory reports. Students must effectively interpret and communicate data based on their understanding of the experiment and the underlying biological concepts. To use lab reports as effective measures of student reasoning, instructors must be able to differentiate between the students’ understanding of the data and their ability to communicate that understanding. When students struggle to write effectively, poor writing hinders instructors from assessing reasoning skills. In our laboratory intensive, lower-division molecular biology course, we found that providing more explicit rubrics and guidelines for writing did little to improve overall writing skills. In 2017, we developed a direct writing intervention that focused on quantitative comparative (QC) statements, i.e., “the rate of x is 3 times faster than the rate of y” (Polito, 2014). We identified four key characteristics of a QC statement (calculation, context, comparison, and clarity) that must be present to provide all the necessary information for the reader to understand the comparison being made by the writer. We developed an annotation scheme (4C annotation) to highlight these elements in student writing and easily identify incomplete or unclear statements. We used the QC statement as the framework to scaffold writing assignments across the entire quarter in our introductory course. We also developed an in-class writing intervention based on 4C annotation to support students’ writing about data. The classroom intervention required 15 minutes of class time and students found it easy to master. We noted a 17% improvement in lab report scores by students who received the 4C annotation intervention compared to students who did not (t343=-7.64, p<0.0001). We also found that we could provide more explicit feedback to students regarding both their writing and their reasoning, resulting in improved grading consistency among instructors and between different writing assignments. We also used the framework of the QC statement to scaffold paragraph level organization of student lab reports. Student improvement in higher order organization of their reasoning was particularly striking as the laboratory experiments became more complex. In summary, we identified 4 key elements to the quantitative comparative statement (4C) and used that framework to support student writing and reasoning at the sentence and paragraph level in written lab reports. The 4C annotation scheme helped us achieve our goal of improving student writing so we can focus on students’ higher order reasoning skills to build compelling conclusions from their data.

Connecting instructor epistemic beliefs to student understanding of science in argument-driven labsLinda Fuselier*, Justin McFadden, University of Louisville[abstract # 90]What teachers believe about knowledge influences how they teach and what their students learn about science (Hasweh 1996; Muis et al. 2010). However, we know little about the complex relationship between instructor epistemic beliefs and student perceptions of science (Mansour 2009; Maclellan 2015). This is especially critical in introductory biology laboratory courses (Duschl & Grandy 2008) where teachers, usually graduate teaching assistants (GTAs), participate in co-constructing knowledge with students during scientific argumentation lessons (Wyse, Long & Ebert-May 2016). We used a case study approach to, 1) describe epistemic understandings of science held by GTAs, and (2) characterize the relationship between GTA epistemology and student understanding of social aspects of science. Our guiding research question was: How do GTAs epistemic beliefs influence their pedagogical choices and impact knowledge construction by a community of learners? The case study included GTAs who teach in a large-enrollment, non-majors biology laboratory course. We used three methods to characterize GTA science epistemic beliefs. We recorded and transcribed student conversations during

23

23

SABER 2017 Abstracts- Short Talks

argumentation sessions and focus group interviews, and examined student written artefacts. We used constructed grounded theory (Charmaz 2006) and inductive coding strategies to reveal contextual clues about each case and to identify emergent themes. We adopted Longino’s Critical Contextual Empiricism (CCE; Longino 2002) as a framework to describe GTA understanding of science epistemology and to analyze the structure of communities of learners in the lab (Bransford & Cocking 1999; Duschl, & Osborne 2002). Results revealed that GTA understanding of science as a social process ranged widely and impacted student perceptions of science during argumentation lessons. GTAs were unified in their belief that science is not an unquestionable authority and that institutional and structural components (e.g., funding) play a role in what is valued in science. GTAs were, in general, unsure of the role that the social plays in constructing knowledge about the physical world and whether observation is neutral. Although, GTAs understood that science had both social and rational components, they did not see these as inseparable and promoted primarily the rational components during argumentation sessions in the lab. Students concentrated primarily on rational components of science and did not perceive science as simultaneously rational and social. Outcomes from this study will be used to identify and encourage pedagogical strategies that enhance understanding of both rational and social aspects of science.

Crossing cultural and institutional boundaries with a health disparities CUREMichele Malotky, Guilford College; Kayla Mayes, North Carolina A&T State University; Gustavo Smith, North Carolina A&T State University; Sherese Mann, North Carolina A&T State University; Mesha Guinyard, North Carolina A&T State University; Sarina Veale, North Carolina A&T State University; H'lois Mlo, Guilford College; Lek Siu, Guilford College; Vung Ksor, Guilford College; Andrew Young, Guilford College; Maura Nsonwu, North Carolina A&T State University; Sharon Morrison, University of North Carolina - Greensboro; Sudha Shreeniwas, University of North Carolina - Greensboro; Kelsie Bernot*, North Carolina A&T State University[abstract # 40]Course-based undergraduate research experiences address institutional needs for increased capacity and equity for student involvement in undergraduate research experiences. Numerous studies have demonstrated that students who participate in high impact practices are more likely to persist and progress in STEM education. As high impact practices, service learning and undergraduate research experiences are based on the theoretical framework of Dewey’s experiential learning through which students learn by “doing” as a social community. Our course included students from two institutions – a public, historically black university and a private, Quaker, liberal arts college. We worked with a 3rd institution on a novel, authentic research project assessing the prevalence of hypertension and associated risk factors in an immigrant/refugee community. We used a number of approaches to assess the impact of this course, including: • Pre/Post exam: direct assessment of scientific literacy • Post indirect assessment of cultural competency (Custom questions in Student Assessment of Learning Gains, SALG) • Direct assessment of group journal club presentations and individual research papers • Indirect assessment of student learning gains (SALG) • Qualitative assessment through a focus group and open-ended questions on the SALG survey • Project Ownership and Networking Survey Mean student performance increased overall from 56.7% to 76.8% (p<0.0001) on the pre/post direct assessment, with largest increases occurring in data analysis (18.5% to 60.9%, p<0.0001). Students reported very strong gains in scientific self-efficacy, cultural competency and ability to work in a group. Furthermore, we have analyzed student networking and project ownership. Together these data suggest that this application of science to a real world problem resulted in significant student learning in traditional scientific skills as well as soft skills. We will discuss changes to course structure as a result of year one findings. The impact of these modifications in year two as well as implications for future directions will be discussed.

Modelling Changes in Expertise in Undergraduate Biology CoursesCody Bekkering*, Michigan State University; Jessica Maher, "Delta Program in Research, Teaching, and Learning"; Diane Ebert-May, Michigan State University; Anne-Marie Hoskinson, Michigan State University[abstract # 172]A common goal for transforming the gateway undergraduate biology curriculum is to educate a generation of individuals who can organize, use, make connections among, and communicate about biological knowledge. Achieving this goal requires students to gain disciplinary expertise. The goals of the present work were to investigate and characterize which aspects of the learning environments in introductory biology courses best predict changes in students’ conceptual expertise. We investigated the roles of learning environments,

24

24

SABER 2017 Abstracts- Short Talks

assessments, and student demographics on observed changes in conceptual expertise in biology. This research utilized the Biology Card Sorting Task (BCST; Smith et al. 2013) to quantify changes in expertise across multiple large-enrollment undergraduate biology courses. We administered the BCST to 751 students enrolled in their first or second semester of introductory biology courses, using both a closed sorting condition with deep conceptual categories provided; and an unframed sorting condition, with student-generated categories. We characterized the instructional environment in the study courses using the Reformed Teaching Observation Protocol (RTOP; Sawada et al. 2002). We coded summative classroom assessments (exams) using the 3-Dimensional Learning Assessment Protocol (3D-LAP; Cooper et al. 2015). We also investigated demographic descriptors. Higher RTOP scores, characterizing student-centered classrooms, were associated with significant gains in conceptual expertise. Women showed higher gains than men, but only on the unframed sorting condition. Upperclassmen and non-first generation collegians also performed better on the framed sorting than freshmen and first generation collegians, respectively. Our results take one of the first looks into the effects of student-interactive teaching on student changes in expertise in undergraduate courses and how these changes are influenced by student demographics.

SUNDAY- Professional Development BEYOND Graduate SchoolCompeting discourses of scientific identity among postdoctoral fellows in the biomedical sciencesRebecca Price*, University of Washington Bothell; Sharona Gordon, Ira Kantrowitz-Gordon, University of Washington[abstract # 10]Postdocs often assume they will become principal investigators (PIs), even though most will not. This discordance brings unique stresses to this stage of a career and profoundly affects the scientific workforce. To learn how postdocs develop their professional identities while dealing with these stresses, we conducted in-depth, semi-structured interviews with 30 postdocs (57% female, 40% male, 3% trans*; 63% white; 20% Hispanic) from biomedical fields at a research-intensive university in the western United States. Interview questions explored how and why the fellows identified as scientists, perceived barriers to success, and long-term career goals. We used discourse analysis, a qualitative method intended to uncover hidden beliefs that guide speech and action in specific contexts, to understand how postdocs identify as scientists. We iteratively reviewed transcripts of the interviews independently and collaboratively to identify implicit and explicit statements about science and scientific careers. We identified two dominant discourses: bench scientist and PI identities. The bench scientist identity comes from directly interacting with the world through experimentation, e.g., “I'm good with my hands… I have that tenacity to make very difficult experiments work.” The PI identity comes from articulating research programs, e.g., “pushing, you know, the limits of knowledge.” Postdocs reported sacrifices that they make to accommodate their careers, including not knowing where or when they would live somewhere permanently, low salaries, and poor work-life balance. Nonetheless, they reported a magical draw to the academy that justified these sacrifices. Furthermore, the bench scientist discourse supports PIs more than postdocs. For example, postdocs who identify as bench scientists spend their time collecting data and writing related manuscripts. These actions promote their PIs, who integrate a series of smaller projects to frame an entire research program. Focusing solely on the bench scientist discourse prevents postdocs from maturing their ability to conduct experiments to the ability to articulate their own, independent research programs. However, being able to articulate a research vision helps postdocs transition to PIs or train for the non-academic positions that most will ultimately achieve. Using this example along with our other results, we formulated a series of recommendations for improving postdoctoral training by institutions and PIs collaborate to: promote a culture that encourages career exploration; help postdocs develop skills that are transferable to a number of careers or specific to their particular career goals; affirm postdocs’ career choices, recording these in Individual Development Plans; and affirm personal choices about quality of life.

Exploring instructor accounts of instructional mindsets in the context of the Scientific Teaching pedagogical frameworkRobert Erdmann*, University of Nebraska - Lincoln[abstract # 80] The Scientific Teaching (ST) pedagogical framework seeks to align teaching practices in science courses more closely with the practices of the scientific enterprise by encouraging data-driven instructional decision making. A

25

25

SABER 2017 Abstracts- Short Talks

key component of ST is a four-step cyclical framework for instructional planning: setting learning goals, determining evidence for learning, planning instruction, and evaluating/reflecting/revising. Investigation of instructor enactment of this framework has been limited. As a result, we endeavored to examine the following research question - how do instructors think about the various steps of the ST framework? We conducted a qualitative analysis utilizing structured interviews conducted prior to and following video observations of discrete units of instruction. Our sample consists of 42 STEM instructors at a large public research institution, with the plurality teaching Biology. We developed a coding scheme for the interviews using structural and thematic coding techniques, and achieved acceptable interrater reliability levels (> 0.80) using the finalized codebook. In addition to analyzing the frequency of categorically-related codes, we utilized the Goodman-Kruskal statistic to screen for the potential correlation of categorically-dissimilar codes across the entire range of potential code co-occurrences. We found that instructor responses regarding learning goals were extremely content/topic-centric, that discussion of assessment techniques frequently defaulted to summative assessment techniques, and that methods of student engagement described tended towards the “traditional” model of instruction, findings that concur with previously published reports. Instructors were generally more satisfied with the student engagement level in their classrooms than they were with the achievement of their classroom goals or with their own teaching. Satisfaction with both teaching and student engagement was in large part determined by personal feelings and levels of student attentiveness/participation. However, the majority of instructors reported that their satisfaction with goal achievement was determined by content coverage. Interestingly, we observed a disconnect between reported satisfaction with goal achievement or learning outcomes and plans to implement course changes, both in the moment and in the future – for example, instructors with stronger self-reported learning outcomes were as likely to express a concrete plan to change instructional approaches as those with weaker learning outcomes. We present these results and associations, as well as others, in an attempt to better characterize the faculty mindsets corresponding to the different steps of the ST framework.

Losing the Lecture: Guideposts on the Path to Reformed InstructionRebecca Reichenbach*, Lisa Montplaisir, North Dakota State University[abstract # 52]It is generally agreed that reform needs to occur in the American Education system at all levels. However, studies continue to show that university faculty are extremely resistant to change even when presented with evidence as to the efficacy of reformed instructional practices, especially active learning techniques. Participants in our project were selected through an application process that targeted those instructors who were not only willing to change but also taught areas of high DFW (“D”, “F”, and “Withdraw”) rates. They agreed to attend periodic workshops and participate in Faculty Learning Community (FLC) meetings over the course of two years. Workshop and FLC topics were chosen from a constructivist viewpoint to scaffold the instructors through better pedagogy. This cohort entered the program with an extreme willingness to change and a favorable opinion towards active learning. The initial instructional technique workshop occurred in January 2016 with additional training occurring in May 2016, August 2016, and January 2017. FLCs are ongoing throughout the two years of participation. We have been monitoring what occurs in the classroom as participants progress through the program. Our research focuses on a sample of the cohort (n = 12) comprised of STEM faculty with large enrollment courses (enrollment > 50) and who taught the same course in comparable semesters (Fall 2015/Fall 2016 and Spring 2016/Spring 2017). To track any changes in practice, we are documenting what occurs in the classrooms using the Classroom Observation Protocol for Undergraduate STEM (COPUS). The COPUS was developed as an instrument to provide feedback to faculty about the active learning occurring in their classroom and has been shown to be able to track shifts in practice. Baseline observations occurred during Fall 2015 and Spring 2016 with implementation being Fall 2016 and Spring 2017. Our Fall 2015 – Fall 2016 COPUS comparisons show significant increases in both “Guiding” oriented instruction (Cohen’s d = 1.67) and “Student Work” (Cohen’s d = 1.78) in our classrooms. The Spring – Spring comparison is still in progress. Additional data streams include analysis of change in course artifacts to reveal alterations in student centeredness and interviews to uncover specific factors of the project that are most and least helpful. We seek to better inform pedagogical instruction in faculty development at other institutions. NSF Grant: DUE 1525056 Keywords: Curriculum Reform/ Institutional Transformation; Faculty/postdoc/graduate student development; Research on effective instruction

26

26

SABER 2017 Abstracts- Short Talks

SUNDAY – Higher Order CognitionDesigning biology education learning environments that incorporate mathematical representations: Centralizing explanation and mechanistic reasoning firstMatthew Lira*, Purdue University[abstract # 192]I will offer an empirically-derived three-prong model of how mathematical representations could address the specific challenges biology students face when explaining phenomena with mechanistic reasoning. Experimental biology has produced information about biological systems at a rate that exceeds our capacity to synthesize that knowledge into a coherent understanding. In response, biologists have forged new partnerships with mathematicians to better understand how biological systems relate. Scientists working at this intersection contend that what makes many problems in biology difficult includes (1) the great numbers of types of objects and objects of each type, (2) the vast arrays of interactions between objects, and (3) the complexity of interactions across different scales (NRC, 2005). These factors also contribute to the learning challenges faced by undergraduate biology students. Biology educators have begun to address related challenges with empirical research on professional development, curricular and institutional change, and instructional experiments. At present, however, we have not built a model of student knowledge that specifies how students might learn biology concepts with mathematics. Echoing NRC-commissioned reports, I contend that the biology must retain primacy when working at the biology-math interface. Research in physics and chemistry education illustrates that pitfalls manifest when students fail to connect the math to the science. In part, these pitfalls stems from students failing to explain how mathematical representations and procedures connect to physical objects in the world. Thus, the general model I present positions explaining at the center of learning in science. But specific to the difficulty of explaining in biology, I argue that any model of student knowledge should correspond to the three challenges noted above. I summarize these challenges as germane to mechanistic reasoning or determining how the low-level organization of a collection of system parts leads to interactions that produce new properties at higher level of organization. Based upon a data corpus from cognitive clinical interviews (n=10) and a written paper and pencil test (n=30), I designed a conceptual model that specifies some of the key challenges that biology students face when reasoning about quantified biological phenomena. The model illustrates that students (1) underspecify the properties of systems, (2) conflate properties of systems, and (3) overemphasize one system property while neglecting others. Nonetheless, when students explain phenomena with mathematical representations they are afforded the opportunity to specify properties, distinguish between them, and emphasize relations between variables and physical quantities. These findings point to the idea that learning environments should adopt mathematical approaches in response to students’ needs and the importance of the learning mechanism of self-explanation.

Exploring student problem solving in geneticsJennifer Avena*, Oscar Whitney, Jenny Knight, University of Colorado, Boulder[abstract # 26]Why do students have difficulty in solving complex biology problems? While previous research has identified differences in problem-solving between experts and novices, research is still needed to address the following questions: What cognitive processes do students use to solve biology problems, and what interventions might help students solve future problems? We examine both of these questions in this study. Students enrolled in an introductory genetics course (N=223) completed a practice assignment that consisted of multiple sets of higher-order genetics questions, with each set containing three questions (Q1-Q3) that address a specific genetics topic. During this assignment, students were provided the chance to receive a content-focused clue for each set of questions. We examined student performance on the assignment as well as on content-related questions on a subsequent course exam. On average, controlling for student performance on Q1, students who received a content-focused clue did not perform better on the assignment or on the subsequent content-matched exam questions than students who did not receive a clue. However, we are still analyzing the data to determine whether a specific subset of students is assisted by these prompts. Importantly, when assessing exam performance, students who were successful in the practice assignment were more likely to answer the related exam questions correctly. Furthermore, practice is generally associated with higher exam performance, as students who completed the practice assignment performed better than students who did not complete the assignment (p<0.05). In addition to examining student performance, we are also examining the procedures students use to

27

27

SABER 2017 Abstracts- Short Talks

solve these problems to identify any differences in processes used by students who successfully versus unsuccessfully solve a problem. In a pilot study (N=65), we assigned a series of problem-solving codes to each student’s written problem-solving documentation and then calculated the proportion of their problem-solving approach devoted to each type of process. When student final answers were separated into correct versus incorrect, we identified several differences (p<0.05). Students who answer correctly more often calculate and recall in questions about probability and more often eliminate possibilities in questions about meiosis and modes of inheritance than students who answer incorrectly. We will explore these potential differences further in the current data set of 223 students. We anticipate that this research will assist us in identifying the process of student problem-solving in genetics as well as potential interventions focused on potentially both genetics content and problem-solving procedures.

Use of Scaffolds to Support Self-Regulated Learning and Metacognition in Undergraduate Biology StudentsJaime Sabel*, University of Memphis[abstract # 83]Undergraduate biology classes must prepare students to engage in an increasingly interdisciplinary field where they will need a foundation of scientific understanding to make informed decisions about the science they will encounter in their careers and everyday lives. To develop this foundation of scientific understanding, students will need to integrate individual concepts into complex biological systems. Providing feedback to students on their progress is an important way to support them in developing robust understanding, but it can be challenging to provide frequent and detailed feedback to individual students, particularly in large, undergraduate classes. However, instructors can support students to engage in self-regulation to monitor their own work, use metacognition to consider their own ideas, and use self-generated feedback to improve their understanding. Instructors can utilize various scaffolds within their courses to provide structure for students to engage in this process. In the first of two studies, I examined the extent to which use of enhanced answer keys and reflection questions could support students to engage in self-regulated learning within an introductory biology course. Students in the course (n=85 participants) completed three surveys throughout the semester that examined how students used enhanced answer keys and reflection questions and the extent to which they engaged in metacognition. A subset of students (n=20) participated in two semi-structured interviews: one before the first exam and one after the second exam. The interviews further explored students’ engagement in metacognition and use of the scaffolds, and allowed for directed instruction on the use of the scaffolds to this subset of students. Findings from this work indicate these scaffolds helped students to engage in metacognition and develop greater understanding of biological concepts. In particular, providing students with instruction on scaffold use led to increased learning gains over students who did not receive instruction. The second study is an extension of the first and focused on how introduction to the reflection questions influences students’ use of the scaffold, how they utilized them to consider their understanding after a practice exam, and the extent to which they engaged in metacognition throughout the course. Findings from these studies have implications for the design and utilization of effective scaffolds and classroom contexts to effectively support introductory biology students as they engage in complex biological concepts and work to increase their understanding. Future work will continue to explore additional scaffolds and to extend these investigations into upper level courses.

SUNDAY- Virtual Learning

Student perceptions of lecture capture video as a facilitator of learning in a flipped biology class.Michael Moore*, Ying Xiu, Penny Thompson, Donald French, Oklahoma State University[abstract # 63]As flipped learning gains popularity as an instructional method for biology courses, there are questions yet to be answered that will help us understand better how students conceptualize online biology learning. Two of these questions are: 1) Do students perceive these videos help them learn biology? and 2) What strategies do students employ when watching these videos? These questions are important because firstly if students do not perceive videos as helping them learn, then students are less likely to watch them. Secondly, we know very little about student video watching behavior. This study explored student perceptions of lecture capture video used in a flipped classroom. Data were collected from an honors biology class (N = 69) during the Fall 2016 semester. From

28

28

SABER 2017 Abstracts- Short Talks

this class, we received 62 responses on a Likert-type questionnaire and 60 responses to an open-ended question about students perceptions of how well the videos helped them learn. Students responded to eight Likert-type questions where they rated, on a 5-point scale from strongly disagree to strongly agree, how much they liked the videos and how much the videos contributed to their learning. The means per question ranged from 2.97 to 4.19, indicating an overall positive response to the videos. Student responses represented a positive perception of the video lectures’ ability to support knowledge acquisition for quizzes (M = 4.19). Conversely, students perceived no benefit from taking quizzes on learning biology (M = 2.97). The open-ended question was analyzed by counting the number of negative, positive, and neutral statements. The three authors coded the responses separately and then discussed each response, adjusting the coding for responses where a consensus could be reached. The Fleiss Kappa rating of inter-rater reliability was .93. The comments were predominantly positive, with most responses containing two or three positive comments. Students who gave positive comments also indicated a wide range of what they perceived to be successful video watching strategies they developed for this course. Students perceived video watching as a driver for active classroom engagement. Negative perceptions of the class centered on lecture video quality, lack of personal presence in the classroom, and a perception of not being taught by the instructor of record. Student representations of video watching strategies help us gain new insights into student engagement patterns in online learning. Additionally, our study helps us understand student attitudes of flipped classroom video use, which will inform future course improvement and research. The Comparison of Learning Gains and Perceptions of Self-Efficacy in a Computer-Simulated vs. “Live” Cell Biology Laboratory ExerciseLara Goudsouzian*, Patricia Riola, Karen Ruggles, Pranshu Gupta, DeSales University; Michelle Mondoux, College of the Holy Cross[abstract # 25]We have developed and implemented a laboratory exercise appropriate for a sophomore or junior-level Cell Biology course. Students are assigned two Saccharomyces cerevisiae cultures, each of opposite mating type. Mating pheromone is added to each culture, and the growth and morphology of the individual mating types is assayed at regular intervals over the course of several hours. Students then prepare graphs of their data, also generating a growth curve to demonstrate that yeast cells will respond only to the presence of the opposite-sex mating factor. Among the lecture topics reinforced by this laboratory are cell signaling, the cytoskeleton, the cell cycle, and the effects of checkpoint activation on growth. Two challenges motivated us to pursue a computer simulation of this laboratory. First, not all of our lecture students are enrolled in the laboratory section of this course. We hypothesized that lecture-only students performing this laboratory in simulation would achieve some or most of the learning gains of students performing the “live” lab. Second, maintaining student engagement during this lengthy and repetitive laboratory assignment is a consistent challenge. We hypothesized that introducing an electronic element to the “live” laboratory would increase student engagement, and, as a consequence, aid learning. Therefore, we entered into a collaboration with a Computer Science seminar course. The students in this course were assigned the task of creating supporting materials for the yeast mating labs, with the Cell Biology professor as the group’s formal “client”. In accordance with the client’s requests, the students generated two different simulations of the laboratory. The two-dimensional (2D) simulation is most useful to support the learning of students who also perform the “live” laboratory. The three-dimensional (3D) simulation is most appropriate for use by those who take Cell Biology lecture without enrolling in the laboratory. Assessment data indicate that both the “live” laboratory exercise and student-designed computer simulations of the yeast mating response lead to learning gains in analytical skills and course content. For some topics, learning gains were greater for students who completed the “live” lab compared to those who only completed the computer simulations. Attitudinal assessment shows that student engagement and students’ perception of their own self-efficacy increased after performing either the simulated or “live” laboratories, as well as both the simulated and “live” laboratories together.

29

29

SABER 2017 Abstracts- Short Talks

Testing the effectiveness of feedback and constraint in an experimental design simulationDenise Pope*, Center for the Integration of Research, Teaching and Learning; Jen Palacio, SimBio; Susan Maruca, SimBio; Kerry Kim, SimBio; Jody Clarke-Midura, Utah State University; Joel Abraham, CSU Fullerton; Eli Meir, SimBio[abstract # 118]Instructors often express frustration with undergraduate student understanding of experimental design (ED). We hypothesized that receiving feedback while designing and carrying out experiments can advance student understanding of ED. We developed a simulation-based module to teach ED, which enables feedback and allows students to design and execute open-ended experiments more quickly than in most real-life experiments. Section 1 of the module introduces principles of ED with examples and formative assessment; in Section 2 students design and execute experiments, collect data and draw conclusions, receiving feedback throughout. Providing meaningful automated feedback on student ED is only possible if the designs can be categorized. An interface allowing for fully open-ended ED (e.g., dragging organisms into plots) results in an infinite number of designs that are intractable to categorize, so we slightly constrained the interface (e.g., allowing addition of organisms from a limited set of options), while still providing flexibility and hundreds of permutations. Recognizing that the constraints themselves may aid student learning by bounding the task’s complexity, we carried out a controlled, mixed-methods study to test the module’s effectiveness and to test for the effects of constraints alone and constraint-enabled feedback. We had 3 treatments: +feedback/+constraints; -feedback/+constraints; -feedback/-constraints. We conducted semi-structured interviews with 42 students recruited from introductory biology classes, randomly assigned to treatment (14/treatment). They completed the module in two sessions and were interviewed pre and post module completion. For the interviews, students were first given a modified EDAT (Experimental Design Ability Test) prompt describing a scenario and asking them to design an experiment. After they wrote a response, we then asked questions to probe their understanding of key concepts in ED (e.g., why change only one variable at a time, why include replication). Two raters used a rubric to score transcripts for these concepts. We also examined students’ actions and answers to questions in the module. We assessed Section 1’s effectiveness with a pre/posttest. Interviewers did not observe the students’ use of the module and were blind to treatment. We are analyzing both interview results and the answers and actions within the module. We will look for any change from pre to post in ED key concepts for all treatments, and compare performance across treatments. Preliminary analyses suggest student understanding improves for some key concepts but not others. Through this study we hope to provide an effective new tool for teaching ED, and provide insight into the optimal design of virtual learning modules.

SUNDAY Formative ToolsHi-Fi Instruction: Characterizing the fidelity of implementation (FOI) of formative assessment and feedbackErika Offerdahl*, Washington State University; Jeff Boyer, North Dakota State University; Melody McConnell, North Dakota State University; Jennifer Momsen, North Dakota State University; Rachel Salter, North Dakota State University; Kurt Williams, North Dakota State University; Lisa Wiltbank, North Dakota State University[abstract # 186]Undergraduate biology transformation efforts continue to gain momentum due, in large part, to the accumulation of experimental studies demonstrating the efficacy of evidence-based instructional practices (EBIPs). Experimental studies demonstrate the viability and efficacy of EBIPS, but the learning outcomes achieved are not always realized when implemented in non-experimental settings. We apply the fidelity of implementation (FOI) framework, commonly used in K-12 curriculum intervention research, to understand enactment of EBIPs “in the wild” in undergraduate biology. The FOI framework suggests that the realized outcomes of instruction are determined by the extent to which critical components of an instructional practice are present. Characterization of the critical components of EBIPs and establishing measures of their enactment is necessary for understanding variation in EBIP outcomes. Formative assessment (FA) (and associated feedback) has been identified in the literature as one such critical component. The overarching goals of this research were to (1) describe FA and feedback during evidence-based instruction and (2) characterize the relationship between variations in FA and student learning outcomes. This presentation focuses on the first goal, and answers the questions: how is FA defined as a critical construct of evidence-based instruction, what are reliable and valid measures of FA implementation, and how is FA enacted during evidence-based instruction? The research context was a large-lecture, intro bio course for majors

30

30

SABER 2017 Abstracts- Short Talks

taught in a SCALE-UP classroom by two instructors, both formally trained in Scientific Teaching and active learning but one with substantially more teaching experience. Guided by the FOI framework, we defined FA and feedback as a critical component through (a) synthesis of the literature, (b) input from FA experts, and (c) classroom observations and instructor interviews. We refined an existing K-12 FA observation protocol (k = 0.82) and used it to enumerate and describe FA cycles. Other measures of FA included (a) recording the format (e.g. clicker, worksheet), Bloom’s level (k = 0.85) and Socratic nature of all FA prompts, (b) describing the frequency and types of student responses, and (c) characterizing the frequency and types of instructor-generated feedback. Our data reveal marked differences in the FOI of FA between instructors with similar pedagogical training and student-centered COPUS profiles, and a measurable difference in patterns of student responses. These variations may explain, in part, the observed differences in student outcomes between experimental and “real world” instructional settings. We discuss implications for faculty pedagogical training and research on EBIPs efficacy.

How instructional decisions influence student buy-in toward and utilization of formative assessmentsKati Brazeal*, "University of Nebraska, Lincoln"; Tanya Brown, University of Colorado; Brian Couch, University of Nebraska-Lincoln[abstract # 180]Formative assessment (FA) techniques, such as clicker questions, in-class activities, and homework, can be powerful tools to improve student learning. However, the efficacy of FAs depends on many factors, including how students utilize them (e.g., their effort level when answering and whether they search the internet for the exact answers or discuss with peers) and how instructors implement them. Instructors face many decisions when using FAs, including what kinds of questions to ask, whether to ask questions before or after the material is covered in lecture, how to provide feedback to students, the grading policy, and other logistical issues. Building on our previous quantitative research on student buy-in, we conducted interviews to qualitatively address how students interact with FAs and how instructional implementation decisions influence student perceptions and utilization of FAs. We interviewed 64 students from eight biology courses ranging from introductory to senior level. We asked each student a set of core questions about their perceptions of FAs used in their course, but the semi-structured interview also allowed for unique follow-up questions based on their answers. The core questions were informed by a FA framework (Black and William 2009) and probed student perceptions about how the FA influences their learning, how they utilize the FA, and how various instructional decisions associated with the FA influence their perceived learning. Thematic content analysis of interview transcripts began with three researchers each reading a set of transcripts to generate a list of codes. One researcher applied these codes to another set of interviews, and all three researchers read excerpts from this set pertaining to each code. Separately, and then through discussion, we generated a list of overarching themes and conclusions extrapolated from the interviews. We found that the question content, amount of feedback, timing, and grading policy of the FAs had apparent influences on student buy-in and utilization of FAs. Students had higher buy-in and engaged in behaviors supportive of learning when they perceived that the FA asked challenging questions aligned with the exams, provided sufficient feedback to correct misunderstandings, was timed in a way that matched their preferred learning style, and had a grading policy that incorporated leniency. Additionally, we found that different implementation decisions had interacting effects on student perceptions and behaviors. This research provides important insights for instructors about how to implement FAs in ways that foster buy-in and encourage student utilization behaviors that support learning.Student perceptions of formative assessment and feedback in an active learning classroomLisa Wiltbank*, North Dakota State University; Kurt Williams, North Dakota State University; Lauren Marciniak, North Dakota State University; Rachel Salter, North Dakota State University; Jennifer Momsen, North Dakota State University; Emily Sederstrom, University of Minnesota; Jeff Boyer, North Dakota State University; Erika Offerdahl, Washington State University; Melody McConnell, North Dakota State University[abstract # 162]In-class formative assessment and feedback are common elements of active learning. Feedback, in particular, is hypothesized to be critical for learning. Little is known about how university students intercept, perceive, and utilize in-class feedback. In this study, we used a series of interviews to investigate introductory biology students’ perceptions of formative assessment and feedback in a high-enrollment class held in an active learning classroom. We hypothesized that many features of the experience with feedback are similar across students, but the unique background of each student likely impacts her or his perception of feedback. We developed cued, retrospective interview protocols through an iterative process, using pilot interviews to refine the protocol. After refinement, 15

31

31

SABER 2017 Abstracts- Short Talks

students were recruited from two sections of a first year biology course. Each student participated in four interviews, for a total of 58 student interviews. Interviews used videos to cue students to class events. During Interview 1, students reported and reflected on their actions and responses during 3-4 instances of in-class formative assessment. Interview 2 took place after the corresponding exam, and students “filled in the gap” of their experience with the material after class through the summative assessment. The paired interviews were repeated once near the end of the semester. Detailed thematic analysis revealed that students generally found in-class feedback to be helpful, and frequently described helpful feedback as a simple confirmation of correct thinking or of a correct answer. However, most students did not translate what they learned in class to post-class study decisions. Most chose to study the way they “usually do”, reporting being unaffected by the class experience except as acting as a cue for questions to expect on the summative assessment. This result indicates a need to consider how instructors can give actionable feedback, as well as helping students to go beyond simply appreciating feedback to using feedback that they receive during in-class activities. During data analysis, we also encountered notable cases of variability of the student experience with formative assessment and feedback. We present three contrasting case studies of this variability, paying particular attention to the experience of a struggling student who eventually withdrew from the course.

SUNDAY VisualizationThe relationship between visual representation and cognitive load: It’s complicated.JESSIE ARNESON*, Erika Offerdahl, Washington State University[abstract # 91]A learner’s capacity for relating information is exceptionally limited; working memory can accommodate only a few elements at a time. Experts, through repeated practice, develop sophisticated mental schema that link multiple elements together, thereby functionally increasing the amount of information they process in working memory. The transition from novice to expert can be characterized as development of such schema. Instructional materials should be designed to support novices’ schema development without overwhelming working memory. To this end, cognitive theory of multimedia learning indicates using visual representations reduces strain on working memory by allowing information processing through both visual and verbal channels. As the use of visual representations (e.g. graphs, diagrams, cartoons) is an integral part of science communication, students interact with visual representations frequently in undergraduate science courses. Multimedia learning theory predicts increased student performance when information is presented visually and verbally. Conversely, cognitive load theory predicts the opposite, that introducing visual representations will decrease performance due to the increased cognitive demand associated with making sense of the representation. A suggested method of reducing cognitive load is to gain familiarity with a particular element through repeated practice. We report on the results of a pilot study conducted in introductory biochemistry to examine whether repeated practice with visual representations on tasks spanning all Bloom’s levels would lead to improved student learning, as measured by both the Introductory Molecular and Cell Biology Assessment (IMCA) instrument and the final course exam. Students in a semester with assessments providing greater visual representation practice did not score differently on the IMCA or on lower Bloom’s level exam questions compared to students in the control semester. A difference was observed, however, on higher order exam items; students who had received more opportunities for practice throughout the semester performed significantly worse (p < 0.001). As higher order exam questions were primarily visual, we cannot determine whether inclusion of visual representation contributed to a steeper drop in performance than might occur with only verbal prompts. We hypothesize the additional cognitive demand of interpreting representations may exceed the limited capacity of the working memory when combined with the cognitive load of higher-order tasks. To test this hypothesis, we compared performance on isomorphic assessment items differing only in presence or absence of a visual representation. Preliminary analysis does not indicate a significant difference between visual and non-visual task performance.

32

32

SABER 2017 Abstracts- Short Talks

A Focus on Polarity: Investigating the Role of Orientation Cues in Mediating Student Performance on mRNA Synthesis Tasks in an Introductory Cell and Molecular Biology CourseJeffrey Olimpo*, The University of Texas at El Paso; Daniel Quijas, Beloit College; Anita Quintana, The University of Texas at El Paso[abstract # 73]The central dogma has served as a foundational model for information flow, exchange, and storage in the biological sciences for several decades. Despite its continued importance, however, recent research suggests that novices in the domain possess several misconceptions regarding the aforementioned processes, including those pertaining specifically to the formation of mRNA transcripts. In the present study, we sought to expand upon these observations through exploration of the influence of orientation cues on students’ aptitude at synthesizing mRNAs from provided DNA template strands. In this context, we defined orientation cues as 5’ and 3’ indicators of polarity represented as part of a DNA template strand or mRNA. Furthermore, these orientation cues were either familiar, in which the DNA template strand was illustrated in the 3’ --> 5’ direction and the mRNA adopted an antiparallel conformation relative to the template strand (as is commonly seen in textbooks and other media), or unfamiliar, in which the inverse was observed. Specifically, a quasi-experimental, quantitative approach was employed to evaluate students’ (n = 45) performance on an 18-item, mixed-format mRNA synthesis task series (MSTs), as well as to examine the mediating influence of participants’ spatial ability (as measured via the Mental Rotation Test) and self-reported confidence in understanding the transcriptional process on their success on the MSTs. A 2 x 2 Repeated Measures ANOVA was performed, in which polarity of the DNA template strand (familiar vs. unfamiliar) and polarity of the mRNA molecule (antiparallel vs. parallel) served as within-subject variables. Data indicated that participants were proficient at solving MSTs when the mRNA molecule was represented in an antiparallel orientation relative to the DNA template, irrespective of whether items were depicted in multiple-choice or open-ended format and regardless of the original polarity of the DNA template strand (F(1,44) = 226.729; p < 0.001; np2 = 0.837). In contrast, participants’ performance decreased significantly on tasks in which the mRNA was depicted in a parallel orientation relative to the DNA template strand, the most common error across this item subset being identification of the parallel mRNA (an average of ~76% of responses). Importantly, participants’ self-reported confidence and their spatial ability were both found to have a moderate, positive correlation with their performance on the MSTs. These data reaffirm the need for future research and pedagogical interventions designed to enhance students’ comprehension of the central dogma, particularly with regard to the importance of polarity in the information flow process.

Perceptual Grouping Affects Biology Students’ Understanding of Evolutionary RelatednessLaura Novick*, Vanderbilt University; Linda Fuselier, University of Louisville[abstract # 21]Theoretical Background Biology students have difficulty interpreting and reasoning with the relational information depicted in cladograms (e.g., Author et al., 2013, 2016; Dees, Momsen, Niemi, & Montplaisir, 2014). We tested Author et al.’s (2016) hypothesis that this difficulty stems at least in part from the Gestalt principles of grouping (e.g., proximity, connectedness). These principles, from perceptual psychology, determine how diagrams are segmented into discrete parts (e.g., Wagemans et al., 2012). Because these parts constitute experienced perceptual objects, which are the units of cognition, the Gestalt principles affect performance on a variety of cognitive tasks (e.g., Coren & Girgus, 1980; Maki, 1981; Stevens & Coupe, 1978). Research Description Students from non-majors (n = 137) and majors (n = 54) introductory biology classes evaluated two similar pairs of cladograms after learning about evolution in class. For example, both cladograms in one pair depicted platypuses as more closely related to kangaroos than to ducks. However, cladogram A showed duck in a different perceptual group than platypus and kangaroo, whereas cladogram B showed these three taxa in the same group: A = (((cardinal + penguin) + duck) + (platypus + (kangaroo + (panda + wolf)))), B = ((salmon + trout) + (duck + (platypus + (kangaroo + (panda + wolf))))). Students were randomly assigned to answer one of three questions asked about each pair of cladograms: Which cladogram (1) provides stronger evidence for, (2) best illustrates, or (3) more accurately depicts the indicated relationships among the three target taxa, or are both cladograms equally good? Students received a score of 0-2 indicating the number of problems for which they (incorrectly) selected the cladogram that had the least closely related of these taxa (e.g., duck) in a different perceptual group than the other two taxa (e.g., platypus and kangaroo). Neither biology class nor question type affected students’ choices of the different-groups cladogram. Students chose the cladograms predicted if they were using Gestalt perceptual

33

33

SABER 2017 Abstracts- Short Talks

grouping approximately twice as often (M = 1.31 out of 2) as expected by chance (0.67). Written explanations indicate that students primarily chose these cladograms because of how the taxa were grouped (M = 41%). These results demonstrate the critical role played by principles of perceptual cognition in students’ interpretations of cladograms. Instruction in tree thinking needs to focus student attention on the influence of these perceptual principles to encourage critical reading of evolutionary relationships presented in cladograms.

SUNDAY- Conceptual ChangeLack of grounding in molecular understanding is a barrier to conceptual understanding of genetic terminologyDina Newman*, RIT; Kate Wright, Rochester Institute of Technology[abstract # 24]Interactive Video Vignettes (IVVs) are an innovative online medium that engage users by incorporating prediction questions, data analysis and comprehension questions in an engaging and accessible way. We developed an IVV to teach students about Mendelian genetics in the context of a human disease. The ideas addressed in the IVV included 1) The relationship between genes, alleles and traits 2) The genetic meaning of dominance 3) Pedigree construction 4) Pedigree analysis of a Mendelian trait and 5) Calculating probability of inheritance. Assessment questions that aligned with each of these concepts were developed and administered before and after students completed the IVV activity. Validation interviews were conducted to ensure assessment question clarity and to gain deeper insight into student thinking (N=14). Pre/post data and IVV usage metrics were gathered from eight classes (22-38 students per class) from three different institutions to investigate student understanding of the concepts linked to the main ideas addressed in the IVVs. Overall, student performance improved on all questions, but not equally for all concepts. Analysis of IVV user metrics revealed that the majority of students could correctly answer questions posed to them during the IVV with one exception: less than one-third of all students (71/222) correctly answered the question asking about the meaning of dominance. Then, despite the fact that the IVV provided accurate explanations to counteract the common misconceptions about dominance, students made little to no learning gains on a related post-test question. Similarly, students struggled on a question about the relationship between genes, alleles and traits despite the IVV directly addressing those concepts. While deeper analysis of the multiple select assessment questions revealed interesting movement within answer choices, overall very few students were able to answer either of the questions completely correctly post-IVV. Analysis of interview transcripts revealed that poor understanding correlated with a lack of grounding in molecular concepts about DNA. For example, students who thought alleles were equivalent to traits could not differentiate between alleles, genes and traits in terms of inheritance and thus performed poorly on other questions. Students who could articulate that alleles are similar DNA sequences of the same gene could provide correct reasoning about inheritance. Findings from this work support the importance of integrating the molecular level when designing activities on DNA-related topics such as genetic inheritance.

Biological variation as a threshold concept: Can we measure threshold crossing?Elise Walck-Shannon, Washington University in St. Louis; Josh Pultorak, University of Wisconsin-Madison; Janet Batzli*, University of Wisconsin-Madison[abstract # 150]Threshold concepts have been valued as a heuristic for learning, however there is limited empirical evidence of ‘threshold crossing’ in the literature. As a candidate threshold concept, biological variation is fundamental to competency in evolution and provides a target for analysis of ‘threshold crossing’. The aim of this project is to attempt to measure ‘threshold crossing’ in learning about biological variation within species using three benchmark features of a threshold concept, namely: 1.) troublesome (counter-intuitive, tacit, ritualized), 2.) discursive (discipline specific language in discourse), and 3.) liminality (wavering uncertainty, discomfort). We conducted semi-structured think-aloud interviews of 29 novice students in a cross-sectional design as they move through a ‘variation-enriched’ curriculum: in pre, current and post-course groups; with 3 expert evolutionary biologists serving as an outgroup. The interviews were conducted in an informal setting and asked students to analyze and explain the variation they observed among ten preserved specimens of Sturnus vulgaris (Common Starling). We developed a three dimensional coding scheme to examine the following aspects of interviewee’s explanations including the troublesome nature of variation, the use of disciplinary language in discourse, and evidence of liminality (e.g., level of disorientation, uncertainty, discomfort). Beginning with discursive analysis, we

34

34

SABER 2017 Abstracts- Short Talks

found that novice students first expand their ‘variation discourse’ while engaged in variation coursework, and then display conformity in language and parsimony in explanations for the post-course group similar to the expert outgroup. Moving to troublesomeness, we recognized the prevalence of essentialist, anthropocentric, and teleological reasoning leading to incomplete or counter-intuitive explanations in the pre-course group, while teleological reasoning was the primary reason for troublesome explanations in current and post-course groups, and even, to a certain extent, in the outgroup as a cognitive ‘short hand’ to explain variation among experts. Regarding liminality, we recognized discomfort and conflicting or disoriented reasoning when students attempted to explain how genetic and cellular variation leads to phenotypic variation—with acknowledgement by students when their thinking became confused. We are now in the process of integrating these three dimensions, and hope to identify clear patterns indicative of threshold crossing among groups. Although we conceive of threshold crossing as an individual’s progression through and beyond a threshold toward a transformed understanding in longitude, we hope this cross-sectional analysis will highlight practical elements of study design and analysis that will be valuable to others seeking empirical evidence for threshold concept learning.

Alignment of students' conceptual models with their explanations of the mechanisms underlying gene expressionElena Bray Speth*, Saint Louis University; Laurie Russell, Saint Louis University; Pranav Konda, Saint Louis University; Alan Parr, Saint Louis University; Adam Reinagel, Saint Louis University[abstract # 156]Calls for reform of biology education stress the key role of scientific practices in teaching for student development of relevant competencies. In an introductory biology course for science majors focused on genetic information flow, storage, and expression at the molecular, cellular and organismal levels, we aimed to foster students’ ability to create, use and interpret multiple types of models, and the ability to articulate scientific explanations. Course monthly exams included a variety of question formats, from traditional multiple-choice questions to constructed-response items requesting students to draw conceptual (box-and-arrow) models and to explain their reasoning. Analysis of students-generated conceptual models has provided insight into development of learners’ cognitive structures, and into “threshold concepts” that are challenging for students; we hypothesize that student-generated conceptual models are also a valid form of assessment that accurately reflects students’ knowledge and understanding of specific concepts in biology. We tested our hypothesis by analyzing students' models and written explanations of the mechanisms underlying flow of genetic information from genes to proteins (transcription and translation). Analysis of a pilot dataset from one course section (n=133) revealed a significant alignment between a traditional metric previously used to evaluate student models (correctness of individual propositions, or phrases, within a model; Dauer et al. 2013, Reinagel and Bray Speth, 2016) and the “explanatory power” of students’ written explanations of the same processes (A. Reinagel, MS Thesis, 2015). We developed an analytical rubric for explanations based on a framework grounded in philosophy of science (Bechtel and Abrahamsen, 2005), which allowed us to parse out students’ written explanations into four categories: complete and correct (mechanistic), correct but incomplete, unclear or mixed, and incorrect. Non-parametric statistical analyses established that students who articulated correct and complete mechanistic explanations of the processes of transcription and translation also constructed significantly more accurate propositions to represent these processes within their models of gene expression. In a second iteration of this study, we collected models and explanations produced on exams by students (n = 250) in two sections of the same introductory biology course; ongoing analysis aims to add robustness to the pilot findings. Triangulation between different sources of assessment data is a key strategy to strengthen research findings (Mathison, 1988). Convergence among the data resulting from our analyses of student models and explanations of the same biological processes represents a first step toward validating conceptual models as accurate representations of students’ understanding.

35

35