Academic Program: Assessment Report 2011-20122.1 Bio 302- Structural and cellular components: Exam...

Academic Program: Assessment Report 2011-2012 Unit/Department: Biology Division: Natural Sciences & Engineering

P&OD 2.11.10 1

Component Description

Program Mission Statement

From your Program Assessment Plan (Statement should articulate the unit/ program mission in support of the institutional mission and include a clearly defined purpose appropriate to collegiate education.)

The mission of the USC Upstate biology program is to provide a biological sciences education in the framework of an undergraduate liberal arts curriculum. As a metropolitan institution, we furnish our students with the knowledge, technical skills, and expertise to contribute to society in the Upstate region of South Carolina and beyond. The biology curriculum prepares students by promoting an understanding of scientific methodologies, concepts, and applications, preparing students for successful careers in research, healthcare, and industry in a global environment.

Goal 1

From your Program Assessment Plan (Describe broad learning outcomes and concepts (what you want students to learn) expressed in general terms (clear communication, problem-solving skills, etc). Goals should focus on discipline-specific outcomes relevant to the program.)

Students graduating with the Bachelor of Science in Biology at USC Upstate will demonstrate: 1. an understanding of the design and conduct of scientific experiments.

Objectives SLO’s (student learning outcomes)

From your Program Assessment Plan (Describes the specific skills, values and attitudes students should be able to exhibit that reflect the broader goals. Objectives (student learning outcomes) transform the general program goals into specific student performance/behaviors that demonstrate student learning and skill development along these goals.

1.1 The student will be able to apply the scientific method when testing hypotheses, designing and conducting experiments.

Assessment Methods

From your Program Assessment Plan (Describes the measure(s) by which the department will know the students are meeting the departmental learning objectives. Includes both direct and indirect assessment. Each SLO should have at least one assessment method.)

1.1. Bio 350 -Scientific method assessment. From the last lab report, students will design and write appropriate hypotheses, methodology/experimental design, results, and conclusions about the hypotheses. Assessed with a grading rubric

1.1. Bio 599 Scientific Method- Assessed using a required oral and written presentation rubric. The rubric for the written presentation is the same as the Bio 350 rubric, and the oral presentation rubric is slightly modified from the written presentation rubric.

Assessment Criteria

Level of achievement you are targeting (Indicate benchmarks, scores on assessment instruments, etc… that would indicate acceptable achievement under your plan)

Acceptable performance will be 70% of students demonstrating success in meeting the SLO's.


P&OD 2.11.10 2

Assessment Results

Actual results and data collected (Make sure to break down data by subgroups (e.g. other campuses or emphases). As appropriate, also include item or category analysis.)

Each biology faculty member evaluated one paper from each course. For Bio 350, 13 of 35 lab reports were assessed; for Bio 599, 13 of 30 papers were evaluated, and 27 oral presentations were evaluated. Figure 1.1 shows that student performance for this year and two previous years consistently exceeded the 70% success criterion, with scores ranging from 85 to 100 percent acceptable (acceptable = satisfactory or excellent).

Action Plan What actions or modifications have been or will be made based on this assessment?

No actions or modification are necessary, as all three measures exceeded the success criterion.

0102030405060708090

100

Bio 350 Written Bio 599 Oral Bio 599 WrittenPe

rcen

tages

Figure 1.1. Student acceptable performance on scientific methods using grading rubrics. Line

indicates 70% acceptable criterion.

2008‐09

2009‐10

2010‐11

2011‐12


P&OD 2.11.10 3

Implementation and Evaluation of Previous Years’ “Action Plan”

How was the action plan identified in the previous year’s report implemented this y ear, and what was the impact?

No action plan identified in last year’s report.


P&OD 2.11.10 4


Goal 2


Students graduating with the Bachelor of Science in Biology at USC Upstate will demonstrate: 2. an understanding of important concepts and methods in the biological sciences.



2.1 The student will be able to describe the structure and function of cellular components. 2.2 The student will be able to discuss and give examples of macro and micro evolution. 2.3 The student will be able to explain and recognize interrelationships and dependencies between abiotic and biotic

components of ecosystems

Assessment Methods


2.1 Bio 302- Structural and cellular components: Exam questions will assess students’ knowledge of these concepts. They will appear on a regular in-class test and their scores will be compared to a pre-test given at the beginning of the semester which tests their knowledge of the basic concepts covered in BIO 101 and 102 that relate to BIO 302.

2.2 Bio 301-Macroevolution assessment methods:

1. A set of Lecture exam questions will assess the student learning outcome.

Bio 301 - Microevolution assessment methods:

1. A set of Lecture exam questions on this topic will assess this SLO

2.3 Bio 301 will be assessed using 1. a set of Lecture exam questions

2.1 – 2.3 Bio 599 The standardized national Major Field Test (MFT) will be used to assess the SLO content/concept knowledge for biology seniors. Results will be used to adjust course and curricular content offerings as required.


P&OD 2.11.10 5

Assessment Criteria


Acceptable performance will be 70% of students demonstrating success in meeting the SLO's related to Bio 301 and 302. For the MFT, acceptable performance will be class summary data at or above the 50th national percentile.


P&OD 2.11.10 6

Assessment Results


Bio 302 results on structural and cellular components this year, as in previous years, were excellent (Figure 2.1). Eight questions were used on the pre- and post -tests. For the past five years, scores on the pre-test were below the 70% success criterion level, but post-test scores increased most years about 30 percentage points to about 80% correct. Post-test results for all five years exceeded the 70% success criterion level. Students’ understanding of this material likely further improves in subsequent elective courses they take.

0

10

20

30

40

50

60

70

80

90

100

2006‐07 2007‐08 2008‐09 2009‐10 2010‐11 2011‐12

Percen

tages

Figure 2.1 Percentage correct answers on pre and post test questions, Bio 302. Line indicates 70% success

criterion.

Pre‐test

Final Exam


P&OD 2.11.10 7

Assessment Results


Bio 301 showed reasonable results for understanding of macroevolution (Figure 2.2). Both lecture and lab macroevolution scores averaged across the two course offerings in the 2011-12 academic year as well as the previous five years nearly met or exceeded the 70% success criterion level. We note that we assess understanding of macroevolution, microevolution and abiotic/biotic factors in a sophomore level course because it is a relevant course required of all students; however, understanding of this material likely further improves in subsequent elective courses they take. Strong understanding of these materials by the students’ senior year is evident in the students’ performance in questions relating to evolution on the MFT exam (see Fig 2.5 below).

0102030405060708090100

Percen

tage

Figure 2.2. Percentage correct answers to macroevolution questions. Line indicates 70% success criterion.

Lecture

Lab


P&OD 2.11.10 8

Both lecture and lab microevolution scores averaged across the two course offerings this past year as well as four of the previous five years all met or exceeded the 70% success criterion level (Figure 2.3).

0102030405060708090100

Percen

tage

Figure 2.3. Percentage correct answers to microevolution questions. Line indicates 70%

success criterion.

Lecture

Lab


P&OD 2.11.10 9

Both lecture and lab abiotic/biotic influences scores averaged across these two course offerings during the 2011-12 academic year exceeded the 70% success criterion level (Figure 2.4). Moreover, we have met or nearly met this criteria for both lecture and laboratory over the previous five years (performance failed to meet this criterion only for lecture in 2009).

Percen

tages


P&OD 2.11.10 10

In the 2011-2012 academic year, our students’ performance on the Major Field Test (MFT) exceeded the national average, with 61% of other institutions achieving a lower average score (Figure 2.5). To put this in context, we created a randomized sample of average SAT scores (or ACT scores converted into SAT score equivalents) for incoming freshman for 42 of the 281 other institutions that also administered the MFT in biology during the same academic year. Out of this sample, only 23% of other institutions had a lower SAT average. This difference in the relative ranking of incoming first year students and graduating seniors indicates that the above-average performance of our students on the MFT is not attributable to them arriving at college better prepared than their peers at other institutions. Instead, the result suggests that our program tends to outperform other programs in increasing students’ knowledge of biology and analytical reasoning.

In the 2011-2012 academic year, our students’ performance also exceeded the national average in all four sub-areas (cell biology, molecular biology and genetics, organismal biology, and evolution and ecology). Performance in cell biology, as well as molecular biology and genetics, is similar to previous years. Performance in evolution and ecology dropped off slightly from a very strong performance in the 2010-2011 academic year while still exceeding our 50% criterion. Performance in organismal biology improved substantially, exceeding our 50% criterion for the first time.

0

10

20

30

40

50

60

70

80

Overall Cell Bio MolecBio/Genetics

Organismal Evol/Ecol

Percen

tile

Figure 2.5 Percentile ranking for summary class data for the four most recent academic years. Horizontal line indicates 50

percentile success criterion

2008‐09

2009‐10

2010‐11

2011‐12


P&OD 2.11.10 11

Results from the MFT may be further divided into nine assessment indicators (Figure 2.6), including eight content areas and analytical skills. In the 2011-2012 academic year, our students scored above the 50th percentile in five of the eight content areas (biochemistry, cell biology, molecular biology and genetics, population genetics and evolution, and ecology). Additionally, they scored well above the 50th percentile in analytical skills. Students scored at the 40th percentile or higher in organismal diversity, animals and plants. We think that their weaker performance in these areas reflects a difference in emphasis in biology education, with some schools maintaining the traditional focus in introductory biology on zoology and botany and other schools, including ours, focusing in introductory biology on cell and molecular biology, as well as organismal biology, population genetics, ecology and evolution. Our students thus have less exposure to organismal, plant and animal biology than at some other institutions.

0

10

20

30

40

50

60

70

80

90

Percen

tile

Figure 2.6. Percentile rankings for assessment indicators, fall and spring semesters. Horizontal line represents 50

percentile success criterion

2008‐09

2009‐10

2010‐11

2011‐12


P&OD 2.11.10 12

We noted last year anomalously low performance in plant and animal biology (Figure 2.6), and we suggested that this might be attributable to changes in the test form between 2009-2010 and 2010-2011. Briefly, last year’s percentile data was determined by the Educational Testing Service in comparison to a different test version, such that more difficult questions with regard to plants and animals may have caused percentile rankings to have plummeted across institutions last year. Now that this test version has been administered for a complete year, data are again being compared within rather than between test versions. We expect that this discrepancy explains the dramatic rebound in scores for plant and animal biology this year. However we do note a general upward trend in performance for plant, animal, and organismal biology between results from two years ago and the most recent year.


1. Changes to our first-year introductory biology sequence in 2008-2009 may be responsible for the better performance in organismal biology this year and strong performances for several other positive results on the assessment indicators from the 9 sub-scores on the MFT. However, the reduced focus on plant and animal biology may have lowered performance for these areas. To address this possibility, we have changed a requirement for the major as of the 2012-13 catalog from requiring all students to take a field course to instead requiring that all students take a field or organismal course. Previously, the only two regularly offered field courses during spring or fall semester were Field Ecology or Plant Taxonomy, both senior level courses, with Plant Taxonomy taught in the spring semester. To fulfill this field-course requirement, many students either took Principles of Ecology instead of Plant Taxonomy or took Plant Taxonomy the semester after taking the MFT. Consequently, the requirement that students take a field course reduced the likelihood that they would take a course related to plants or animals in a time frame that would impact their MFT scores. The revised requirement should boost our enrollment in organismal courses, including those related more directly to plant biology and animal diversity The new requirement is in place for students matriculating in the 2012-2013 academic year, so it will take a few years to determine whether it further boosts students’ performance in the plant and animal subsections of the MFT. 2. Although we are meeting or nearly meeting our criterion of 70% success for students showing understanding of aspects of cell biology, evolution and ecology (Figures 2.2 – 2.4 above), we are working towards the goal of all students showing competency in these areas. One change across our curriculum that we have instituted in the 2012-13 catalog is that students must earn a grade of C or better by their second attempt at any major’s biology course in order for the course to fulfill a requirement for the major (this policy had been in place for one year, but withdrawals had not previously counted as an attempt, so the policy had limited effect). In last year’s assessment report, we documented a trend that students who retook courses typically scored similarly in each repetition. Because the students are assessed each semester they take the course, these underachieving students are disproportionately represented in our sample. The presence of multiple repeaters in a course also has the potential to detract from a productive learning environment, resulting in lower outcomes for classmates. Finally, student awareness of the new retake should motivate students to work toward a stronger performance during their first course attempt. We will monitor whether this policy improves student performance in the 2012-13 academic year compared to previous years.


P&OD 2.11.10 13



1. The improved performance in organismal biology this year may be attributable to changes we instituted in our introductory biology sequence in Spring 2009, such that then first-year students would most likely have taken the MFT exam in the 2011-2012 academic year. We will monitor scores in organismal biology in future years to determine whether this improvement is anomalous or represents a genuine change. 2. Our action plan last year stated that we would enforce prerequisites for senior seminar, the course in which students take the MFT exam. Previously, students enrolled in the course despite being inadequately prepared, and this harmed their performance in both the course and their MFT exam. Last year, we asked all faculty to rigorously check prerequisites for all students in senior seminar, such that seats would go to the qualified students. We do not have direct data on the impact of this change because we do not know the identities of students lacking adequate prerequisites who were deterred from signing up for senior seminar. However, performance on the MFT improved for seven of the nine assessment indicators, suggesting that the change may have been beneficial.


P&OD 2.11.10 14


Goal 3


Students graduating with the Bachelor of Science in Biology at USC Upstate will demonstrate: 3. knowledge of and skill utilizing appropriate laboratory techniques in the biological sciences.



3.1 The student will demonstrate certain laboratory techniques (such as light microscopy, gel electrophoresis, population sampling, aseptic techniques, micro and macro pipetting).

Assessment Methods


3.1 Bio 350 - Laboratory Techniques Assessment: Laboratory technique pertinent to genetics experimentation will be graded. Lab book will also be assessed.

Assessment Criteria



Assessment Results


During the past year students were successful in demonstrating laboratory techniques. Of the 42 students evaluated in 2011-2012 academic year, success on the four molecular techniques ranged from 76% to 100%. All students successfully demonstrated the first three molecular techniques. About 92% of students correctly sorted fruit flies, and 100% successfully demonstrated aseptic technique. These values are in line with results from previous years (Figure 3.1). The technique with the lowest success rate (but exceeding the success criterion level) was DNA digestion and electrophoresis. The lower success rate for this exercise results from it being a multistep process, such that even low error rates in individual steps are compounded, resulting in a higher error rate overall. Many students gain further competency in electrophoresis in Molecular Cell Biology (Bio 550) after completing this course.


P&OD 2.11.10 15

Comment [BM1]: George indicated a problem here, but field doesn’t display properly on macs...

0102030405060708090

100

Percen

tages

Figure 3.1. Percentage of students with appropriate lab technique over the three most recent years. Line

indicates 70% success criterion.

2009‐10

2010‐11

2011‐12


P&OD 2.11.10 16


We note improvement in student success with DNA digestion and electrophoresis, and this success may result from greater instructor attention to teaching this skill (see previous action plans below). However, this technique is still the most challenging for students. Related techniques are taught in the prerequisite course Introduction to Cell and Molecular Biology (BIO 302), such that students with this background should perform well on this exercise when they are assessed in Bio 350. The Bio 302 prerequisite has not previously been enforced, and it is possible that students not showing proficiency lack this prerequisite. This year, we will collect individual data on students with and without the prerequisite to determine if this variable affects student proficiency with this and other laboratory techniques. If it does, the subsequent action will be to more rigorously enforce the prerequisite.



Our previous action plan called for the instructors for Genetics to provide more thorough instructions, guidance, and feedback in the DNA digestion and electrophoresis exercise. This change may explain the 10 percentage point improvement observed in student success with this technique.


P&OD 2.11.10 17


Goal 4


Students graduating with the Bachelor of Science in Biology at USC Upstate will demonstrate: 4. an ability to critically analyze, evaluate and interpret scientific information.



4.1 The student will be able to access, recognize, summarize and critically analyze primary literature.

4.2 The student will be able to apply and interpret descriptive and inferential statistics

Assessment Methods


4.1 Bio 599 - This SLO will be assessed using a required research paper and an oral presentation that will be evaluated by a rubric.

4.2 Bio 350 – This SLO will be assessed using a required research paper that will be evaluated

by a rubric.

Assessment Criteria



Assessment Results


In Bio 599, the goal on scientific information was evaluated with both written and oral presentations. The rubric for written reports contained a section on Critical Analysis of Primary Literature (Figure 4.1). Of the 13 papers evaluated in 2011-2012, 92% were judged acceptable. This 92% is a continued improvement over previous years.


P&OD 2.11.10 18

0

20

40

60

80

100

2008‐09 2009‐10 2010‐11 2011‐12

Percen

tages

Figure 4.1. Percentages of senior seminar papers that acceptably summarize and provide critical analysis of

primary literature. Line indicates 70% success criterion.


P&OD 2.11.10 19

We assessed the oral presentation skills of 27 students in senior seminar (Bio 599). The rubric contained two relevant items: Scientific Methods and Understanding of Data Analysis. For the past four years, in both items, students were rated as acceptable (satisfactory or excellent) over 80% of the time, well above the 70% success criterion (Figure 4.2). In the 2011-2012 academic year, we achieved greater than 90% success for both measures, and we note that the few students who did not reach the acceptable level did come very close to reaching it.

0

10

20

30

40

50

60

70

80

90

100

Scientific Methods Understanding of DataAnalysis

Percen

tages

Figure 4.2. Percentages of senior seminar oral presentations that were satisfactory or excellent in scientific information

areas. Line indicates 70% success criterion.

2008‐09

2009‐10

2010‐11

2011‐12


P&OD 2.11.10 20

Descriptive and inferential statistics usage evaluated in Bio 350 showed very strong student performance (Figure 4.3). Over 80 percent of papers the past four years were judged to be satisfactory or excellent, well above the 70 percent success criterion.


No actions or modification are necessary, as all measures met or exceeded the success criterion. Senior seminar does incorporate revision of unsatisfactory papers, and review of primary literature is a component of some upper level courses. No “universal” changes were made to other biology courses.

0

1020

3040

50

6070

8090

100

Descriptive & Inferential StatisticsPe

rcen

tages

Figure 4.3. Percentages of genetics lab reports that were satisfactory or excellent in use of statistics. Line

indicates 70% success criterion.

2008‐08

2009‐10

2010‐11

2011‐12


P&OD 2.11.10 21



No action plan last year for this goal.


P&OD 2.11.10 22


Goal 5


Students graduating with the Bachelor of Science in Biology at USC Upstate will demonstrate: 5. effective communication skills.



5.1 The student will prepare oral and written reports in a standard scientific format.

Assessment Methods


5.1 Bio 350 - Written reports in scientific format: The last lab report, structure of paper and appropriate use of the different sections of paper will be evaluated. 5.1 Bio 599- This SLO will be assessed using a required research paper and an oral presentation

that will be evaluated by a rubric. 5.1 Bio 599- This SLO will be assessed using a required research paper and an oral

presentation that will be evaluated by a rubric.

Assessment Criteria



Assessment Results


The evaluation of the lab report in Bio 350 assessed three aspects: organization, mechanics, and references (Figure 5.1). For 2011-2012, two measures were above the 70% success criterion, with mechanics 1% shy. In 2008 only mechanics was below the success criterion level, but in both subsequent years the 70% success criterion was exceeded.


P&OD 2.11.10 23

0102030405060708090

100

Organization Mechanics References

Percen

tages

Figure 5.1. Percentages of genetics lab reports that were satisfactory or excellent. Line indicates70%

success criterion.

2008‐08

2009‐10

2010‐11

2011‐12


P&OD 2.11.10 24

The evaluation of Bio 599 written papers also assessed the same three aspects as Bio 350: organization, mechanics, and references (Figure 5.2), and showed trends similar to Bio 350. Both organization and references were stronger areas than mechanics, but mechanics improved somewhat since last year and is again above our success criterion. Organization continues to be the strongest area, with acceptable senior seminar papers at 100 percent the past three years.

0102030405060708090

100


Percen

tages

Figure 5.2. Percentages of senior seminar papers that were satisfactory or excellent. Line indicates 70%

success criterion

2008‐09

2009‐10

2010‐11

2011‐12


P&OD 2.11.10 25

The Bio 599 oral presentations (Figure 5.3) showed a high level of performance in the same three areas evaluated in the written presentation of Bio 350 and Bio 599. Between 81 and 100 percent of the evaluated presentations (35 in 2008-09; 34 in 2009-10, 26 in 2010-11, 27 in 2011-12) were judged satisfactory or excellent. Thus, performance in all three areas during the 2008-2012 academic years exceeded the 70% success criterion. We also note that presentations falling short of the criterion were very close to meeting it (usually scoring a 6 on a rubric for which a score of 7 was deemed acceptable).

0

10

20

30

40

50

60

70

80

90

100


Percen

tages

Figure 5.3. Percentages of senior seminar oral presentations that were satisfactory or excellent.

Line indicates 70% success criterion.

2008‐09

2009‐10

2010‐11

2011‐12


P&OD 2.11.10 26


Our students’ greatest difficulty in meeting SLO 5.1 (preparing effective written and oral reports) is with the mechanics of writing, though they did meet or nearly meet our three relevant success criteria this year. Nevertheless, to help them improve their mechanics in writing, we have made changes to their 300-level genetics course and 500-level senior seminar. In Genetics (Bio 350), the instructor has this year (Fall 2012) introduced a new preparatory exercise in which students write several portions of one lab report and receive feedback prior to writing a complete lab report, which we use for assessment purposes. This additional practice and feedback should better prepare students for writing successful lab reports.

For senior seminar (Bio 599), we have consulted with the director of the Writing Center about materials with which to teach scientific writing. Limited staffing at the writing center precludes their staff from providing in-class instruction in all of our senior seminar sections. However, a link to an on-line seminar from the Writing Center was provided along with the associated presentation file. The material includes common problems with word choice, grammar and style. We will use this material in senior seminar this year, as well as related material developed by a Biology faculty member last year. We will monitor whether mechanics in student papers further improves.



Last year, our action plan included devoting more time to grammar and mechanics in the senior seminar course. This change was implemented by multiple instructors during the 2011-12 academic year. This change may account for the improvement in mechanics in senior seminar this year. The instructor has offered to share his materials with other instructors, such that there can be uniformity in the approach to teaching this subject. We are also working to further improve our writing instructions with help from the USC Upstate Writing Center (see action plan above)

Academic Program: Assessment Report 2011-20122.1 Bio 302- Structural and cellular components: Exam...

Documents

Transcript of Academic Program: Assessment Report 2011-20122.1 Bio 302- Structural and cellular components: Exam...