Eval an Inquiry-Based tics Course With Q OnLine JOST Version 2008

8/2/2019 Eval an Inquiry-Based tics Course With Q OnLine JOST Version 2008

1/12

Evaluating an Inquiry-based BioinformaticsCourse Using Q Methodology

Susan E. Ramlo1 , David McConnell2, Zhong-Hui Duan3 and Francisco B. Moore4

Published online: 4 March 2008

Abstract Faculty at a Midwestern metropolitan public university recently developed a course onbioinformatics that emphasized collaboration and inquiry. Bioinformatics, essentially theapplication of computational tools to biological data, is inherently interdisciplinary. Thus part ofthe challenge of creating this course was serving the needs and backgrounds of a diverse set ofstudents, predominantly computer science and biology undergraduate and graduate students.

Although the researchers desired to investigate student views of the course, they were interestedin the potentially different perspectives. Q methodology, a measure of subjectivity, allowed theresearchers to determine the various student perspectives in the bioinformatics course.

Keywords Bioinformatics - Q methodology - Assessment - Subjectivity - Course evaluation

Introduction

The rapidly emerging discipline of bioinformatics studies biological processes and genomic and

proteomic data using analytic theory and practical tools of computer science, mathematics andstatistics. Bioinformatics techniques allow biologists to sift through enormous volumes of dataand discern biological patterns that have long been too intricate to identify. Advances ingenomics, proteomics, and bioinformatics will impact society at the biomedical, economic,cultural, and political. A national need for a workforce knowledgeable in bioinformatics has thusemerged and created full bioinformatics programs, single dedicated courses, and summerworkshop programs.

Journal of Science Education and Technology

Springer Science+Business Media, LLC 2008

10.1007/s10956-008-9090-x

(1) Department of Engineering & Science Technology, The University of Akron, Akron, OH, USA

(2) Department of Geology, The University of Akron, Akron, OH, USA

(3) Department of Computer Science, The University of Akron, Akron, OH, USA

(4) Department of Biology, The University of Akron, Akron, OH, USA

Susan E. RamloEmail: [email protected]

Page 1 of 1210.1007/s10956-008-9090-x

3/5/2008http://www.springerlink.com/content/n0tq77wjg013402v/fulltext.html


2/12

Cohen (2003) previously detailed guidelines for creating a course in bioinformatics. As Cohensuggested, the course created within this study, at a mid-sized public university in the Midwest,involved collaboration among faculty in biology and computer science and targeted preparingstudents for career opportunities in the fast growing biotechnology industry or for advancedgraduate study in bioinformatics. This was a single dedicated course in bioinformatics thatadapted existing course materials from the National Center for Biotechnology Information(NIH), the Human Genome Project (DOE), and other academic institutions. Inquiry-basedteaching strategies were incorporated into both lecture and lab settings of this 4credit hourupper-division undergraduate/graduate bioinformatics course to ensure effective learning.

The bioinformatics course was team-taught by a computer scientist and an evolutionarygeneticist. The course used hands-on projects to demonstrate the capabilities and limitations ofcurrent techniques. Instructors developing a course that draws students from computer science,biology, and other related fields must consider different student perspectives, skill sets, andknowledge-bases. Thus, the instructors sought to adapt the course design on the basis of a courseassessment program that included multiple opportunities for student feedback. This paperreviews how this data was collected and used to guide the evolution of the course through two

iterations of this assessment.

Although the Q methodology was primary in determining students perspectives of the course inorder to make improvements, similar applications of Q methodology were not found within theliterature. However, Q methodology has been used to determine views in other educationalsituations such as evaluating a university faculty reading circle program (Ramlo and McConnell2008) and for course evaluations (Jurczyk and Ramlo 2004). However, Q methodology has beenused to determine perspectives related to bioinformatics in a study that identified and categorizedthe opinions of primary care physicians and medical students related to their reasons foracceptance of and/or resistance to adapting information technologies in the health care workplace(Valenta and Wigger1997).

Methodology

William Stephenson developed Q methodology, or Q, as a means of measuring subjectivity(McKeown and Thomas 1988; Stephenson 1955). Q allows participants to provide theirperspectives by sorting items, typically statements related to the topic, into a forced Gaussiandistribution determined by the researcher. The statements selected are called the Q sample. Thesestatements represent the universe of communicability from which they have been drawn. In thisstudy, participants sorted a Q sample that consisted of 29 statements into the grid shown in

Fig. 1. The sorting is done based upon a condition of instruction. For this study, participantssorted based upon their views of the bioinformatics course. Thus, the sorting process issubjective such that each item is judged relative to the others by the participant performing thesort. Because Q measures personal opinion regarding the statements related to a topic, validity isnot a consideration (Brown 1999). Similarly, operational definitions are not meaningful in Qbecause the researchers view of the items is independent of the determination of the views of theparticipant (Brown 1980).

Page 2 of 1210.1007/s10956-008-9090-x



3/12

Fig. 1 Sorting grid used in this study for the distribution of 29 statements

After the participants have completed the sorting process, the sorts are analyzed via factor

analysis which uses correlation to group people with similar views into factors. Thus, in Q, eachfactor represents a distinct perspective within the group and the terms factor and perspective arethus used interchangeably within this article. Although each sort is subjective, the factorsdetermined in Q are grounded in concrete behavior and are typically reliable and replicable(Brown 1980). The analyses in Q methodology include a number of tables. Among these tablesis a representative Q sort for each factor. These representative sorts are used to interpret eachfactor in conjunction with other participant characteristics such as demographic information andtables comparing the factors based on item distribution. In addition, statements that representconsensus among the factors are also reported within the Q analyses (Brown 1980; McKeownand Thomas 1988).

Analyses

All Q sort data were entered into PQMethod (Schmolck2002), a program designed specificallyfor the analysis of Q sorts. Demographic information, collected with the Q sorts, was used tocode each participants sort as shown in Tables 1 and 2. This information included the studentsmajor (biology, computer science, or anthropology), class level (junior, senior, or graduatestudent), and sex (male or female). In addition to the demographic information, students self-ratings, on a 110 scale, for understanding of biology and computer-science concepts wereincluded in the individual identifications, respectively.

Table 1 Factor matrix with an X indicating a defining sort for spring 2005 cohort

Sorter # QSORTFactor loadings

1 2

1 BSM92 0.2492 0.4762X

Page 3 of 1210.1007/s10956-008-9090-x



4/12


5/12

Note: The sorter identification indicates major (1st letter for biology, computer science or anthropology),academic level (2nd letter for (junior, senior, graduate student), sex (3rd letter as male or female), selfreported rating (on a scale of 110) of understanding biology (1st number) and computer science (2ndnumber)

Data were collected during two different offerings of the bioinformatics course, thus resulting ina spring 2005 cohort and a spring 2006 cohort. The researchers used independent group t-tests tocompare these two cohorts based upon demographic variables and self-reported conceptual

understanding. Computer science and non-computer science (predominantly biology) groupswere compared with t-tests. From the t-test results comparing the computer science majors andnon-computer science majors, computer science majors were significantly correlated with havinga high self-selected score for computer science conceptual understanding and a low self-selectedscore for biological conceptual understanding. Because most non-computer science majors werebiology majors, it is not surprising that the opposite was true for these students; they had highself-selected scores for understanding biology concepts and low self-selected scores forcomputer science concepts.

Comparisons between the two cohorts, spring 2005 and spring 2006, indicated no statisticallysignificant differences for major, academic level (e.g. senior versus graduate student), or sex. Inaddition, no significant differences were found among the two cohorts self-selected conceptualunderstanding of biology or computer science. Although these cohorts were similar based uponthese variables, the Q analyses were run separately for each group since the purpose of this studywas to compare the two groups views of the course. The first cohort, spring 2005, contained 14participants and the 2006 cohort consisted of 19 participants.

Recommended procedures for analyzing the Q sorts were used for the analyses in this study. Forinstance, factors were extracted using the centroid method because it is the preferred factorextraction method for Q methodology (Brown 1980; Stephenson 1955). The maximum numberof factors was allowed for the first extraction in order to investigate how many factors existedwithin the data. For both cohorts, only two factors emerged with sorters flagged, based upon analgorithm within the software, as representative of those factors. Recall that each factor

represents a distinct perspective that exists among the group of sorters. Only those sorts flaggedfor a factor are used for the analyses.

Results & Discussion

15 CSF68 0.5040X 0.4858

16 BEGM78 0.4257 0.4896X

17 BSF94 0.6215X 0.2351

18 BGM83 0.6905X 0.0015

19 CGM610 0.0882 0.6733X

% expl.Var. 24% 18%

Page 5 of 1210.1007/s10956-008-9090-x



6/12

For the spring 2005 cohort, the first factor consists of seven students and the second factorconsists of four students. Each of these factors consisted of both computer science and biologystudents as well as both undergraduates and graduate students. Each factor was represented bystudents with a range of self-rated conceptual understanding in biology and computer science.The remaining three students were not defined by a single factor. As stated earlier, each factorrepresents a unique perspective of the bioinformatics course which is revealed in the griddistribution of statements for each factor. Tables 3 and 4 contain the statements with the highestand lowest grid-positions for Factor 1.

Table 3 Bioinformatics spring 2005Factor 1top 6 most-agree statements

Table 4 Bioinformatics spring 2005for Factor 1bottom 6 least-agree statements

The top most-agree statements for Factor 1 seem to reiterate a concern regarding the learning ofprogramming. The view of bioinformatics comes out in statement 24, in +3 grid position, whichstates that I feel that I could contribute to the development of the field of bioinformatics. Thus,these students seem interested in bioinformatics but weak in the area of programming. Statement7 in the 5 grid position (the most extreme least-agree statement), indicates that Factor 1students view of bioinformatics has changed dramatically due to the course. However, the otherleast-agree statements indicate how the course was organized was a problem for studentsrepresented by this factor. Yet these Factor 1 students were not unhappy with the availability ofinstructors nor did they feel that what was missing from the course was spending more time withthe instructor. Instead, as the strongest most-agree statements indicate, students who represent

No. StatementGridposition

15 Having the class material available online helped me learn +5

17 We should have had a separate textbook on Perl or basic programming +4

10 We should have had spent more time learning how to write programs +4

24I feel that I could contribute to the development of the field ofBioinformatics

+3

11 Interactions with my group during class helped me understand concepts +3

9 We should have used computers more in class +2

No. StatementGridPosition

25 This course was well organized

229 The exams did not reflect the material we covered during class 3

28 The late meeting time for the class made it difficult for me to pay attention 3

21 The instructors were not available outside of class time 4

14Class time would have been better spent with the instructor lecturing ratherthan having the students work in groups

4

7 My concept of bioinformatics has not changed much over the semester 5

Page 6 of 1210.1007/s10956-008-9090-x



7/12

this factor, both computer science and biology majors, wanted more time practicing and moreinformation on programming, possibly working with other students (as represented by statement11). Thus, this factor could be named Could contribute to bioinformatics but need basicprogramming skill development.

The six highest and lowest grid-position statements of Factor 2, from the spring 2005 cohort, aredisplayed in Tables 5 and 6. These statements with the strongest grid positions demonstrate thatstudents on this factor felt very dissatisfied with the bioinformatics course. They did not like howthe course was organized nor did they like working in their groups (possibly because there was amixture of biology and computer science students, as suggested by statement 26 at the +4 gridposition). Three of the least agree statements, 23 (4 grid position), 22 (at 3), and 24 (also at3), indicate that these students have no desire to take another bioinformatics course orinvestigate this field further. This view, therefore, was named Negative view of bioinformaticsas presented in the course. However, this is a bipolar factor with two of the four students whoare represented by this factor with negative factor loadings while the others had positiveloadings. In other words, the two students with positive factor loadings, both biology majors,have a view as described above. Yet, the two students with negative factor loadings, both

computer science majors, have a completely opposing view where the negative grid positionswould then be positive and visa versa. In other words, those with negative loadings on Factor 2do not represent a third factor but, instead, a Positive view of bioinformatics that is in directopposition to the view of those with positive loadings.

Table 5 Bioinformatics spring 2005for Factor 2top 6 most-agree statements



10 We should have had spent more time learning how to write programs +5

26 For many of the topics it would have been better to have separate lectures forcomputer science and biology students +4

6 There was too much material to learn in this course +4

18We should have had a separate textbook or readings that introduced criticalbiological concepts discussed in the class

+3

17 We should have had a separate textbook on Perl or basic programming +3

27The students should have had more opportunities to give feedback on how theclass was taught

+2


13 People in our group worked well together 2


3

I feel that I have a strong understanding of Bioinformatics after taking this

Page 7 of 1210.1007/s10956-008-9090-x



8/12


9/12


The spring 2006 cohorts Factor 1 represents the views of 11 of the 19 participants. Three of thetop six most-agree statements for this factor, shown in Table 7, had to do with the studentgroups. This factor views the group work as a very important component to their learning in thecourse. This is further substantiated by statement 14s 3 grid position. Yet the studentsrepresented by this factor did not have interest in enrolling in a graduate program inBioinformatics (statement 23 at the 4 grid position). Thus, this factor was named Positive viewof the course but no interest in pursuing Bioinformatics.

The eight remaining participants are represented by Factor 2 in the spring 2006 class. Thesestudents were biology, biomedical engineering, and computer science majors. Interestingly, thesestudents had a very positive view of the Bioinformatics field. They are interested in graduatestudy in Bioinformatics (statement 23 at grid position +5) and believe they could contribute tothis field (statement 24 at +4). Possibly these views are due to their learning of bioinformatics inthis course (statement 22, +3, and statement 7, 3). However, unlike Factor 1 from this class,these students did not have a positive experience working in groups (statement 13, 2 position).

Yet they were satisfied with the level of computer interaction and help they received (statements1, 9, and 10) and the amount of material covered in the course (statement 6, 5). Therefore, thisfactor was named Want to pursue bioinformatics.

The 2005 cohort factors indicated a lot of concern about the learning of computer scienceconcepts and programming. Yet this seeming frustration with the programming aspect of thecourse does not appear in either factor from the 2006 cohort. In addition, Factor 2 from the 2006cohort indicates that eight students were willing to pursue graduate work in bioinformatics. In

23 I would be interested in enrolling in a graduate program in Bioinformatics +5


+4

25 This course was well organized +4

22 I feel that I have a strong understanding of Bioinformatics after taking thisclass +3

1The instructors worked hard to help biology students learn computer scienceconcepts

+3

4 I learned much of what I needed for this class from the lecture +2

No. Statement Grid position

13 People in our group worked well together 2

7 My concept of bioinformatics has not changed much over the semester

3

9 We should have used computers more in class 3

27 The students should have had more opportunities to give feedback 4

10 We should have had spent more time learning how to write programs 4

6 There was too much material to learn in this course 5

Page 9 of 1210.1007/s10956-008-9090-x



10/12

contrast, Factor 1 from the 2005 cohort indicated the ability to contribute to the bioinformaticsfield yet they did not indicate an interest in pursuing bioinformatics further. From the 2005cohort, only the two computer science students with significant negative loadings on Factor 2indicated an interest in pursuing bioinformatics in graduate school. Yet for the 2006 cohort,Factor 2 represented eight students, both computer science majors and not, who are interested inpursuing bioinformatics in graduate school.

Conclusions

Q methodology allowed the researchers to evaluate students views regarding their experiencesin bioinformatics within an inquiry based bioinformatics course taken by both undergraduate andgraduate students. This method allowed for multiple voices to be heard within this context andallowed for comparison of views between two different offerings of the course separated by oneyear. Other quantitative methods, such as Likert scale surveys, would not have allowed such

detailed results (McKeown 2001). Qualitative methods such as individual interviews would havebeen extremely time-consuming.

The various views determined were not based upon student major or some other demographiccharacteristic. Differences among the two sections of the course cannot be attributed to initialstatistical differences because no statistically significant differences among the demographicvariables were found between the two groups. Thus, the researchers were able to examine howchanges made after the initial bioinformatics course affected students views of the course andbioinformatics in general.

For instance, based upon the Q results and the open ended questions from our firstbioinformatics course offering, both biology and computer science students had concerns aboutunderstanding and learning the computer science portions of the bioinformatics course.However, these concerns were not represented in the 2006 student views determined with Q.Thus, many changes in the course prior to the 2006 offering addressed the concerns that emergedfrom the 2005 student Q sorts. For instance, faculty assigned student groups to make them morediverse academically and more efficient. In addition, the labs were reorganized so that computerscience labs were covered at the start of the 2006 semester. Similarly, more research aspects ofbioinformatics were introduced into the labs and projects for 2006. These changes to the courseappear to have resulted in less concern about the computer science aspects of the course andmore positive views of bioinformatics both academically and as a potential field of study. Thisinformation may be helpful to others who are developing bioinformatics courses that crossdisciplinary boundaries. In addition, similar applications of Q methodology may assist others

developing or evaluating courses that are also inherently interdisciplinary. Thus, by investigatingstudent views with Q methodology, the researchers were able to reveal how course changesaffected students views of the course and the field of bioinformatics.

Acknowledgements This research supported by an NSF CCLI grant #0410727

AppendixList of the 29 Q sample statements

Page 10 of 1210.1007/s10956-008-9090-x



11/12

References

1. The instructors worked hard to help biology students learn computer science concepts.

2. The instructors worked hard to help computer science students learn biology concepts.

3. I learned much of what I needed for this class on my own.

4. I learned much of what I needed for this class from the lecture material.

5. I learned much of what I needed for this class from the lab and homework assignments.

6. There was too much material to learn in this course.

7. My concept of bioinformatics has not changed much over the semester.

8. Bioinformatics is a lot different than what I thought it was when the class began.

9. We should have used computers more in class.

10. We should have had spent more time learning how to write programs.

11. Interactions with my group during class helped me understand concepts.

12. Our group met outside of class several times.

13. People in our group worked well together.

14. Class time would have been better spent with the instructor lecturing rather than having

the students work in groups.15. Having the class material available online helped me learn.

16. The instructors tried to use everyday terms and simple examples whenever possible.

17. We should have had a separate textbook on Perl or basic programming.

18. We should have had a separate textbook or readings that introduced critical biologicalconcepts discussed in the class.

19. The textbook had almost all of the information necessary for this course.

20. I completed all or almost all (90 + %) of the reading assignments.

21. The instructors were not available outside of class time.

22. I feel that I have a strong understanding of Bioinformatics after taking this class.

23. I would be interested in enrolling in a graduate program in Bioinformatics.

24. I feel that I could contribute to the development of the field of Bioinformatics.

25. This course was well organized.

26. For many of the topics it would have been better to have separate lectures for computerscience and biology students.

27. The students should have had more opportunities to give feedback on how the class wastaught.

28. The late meeting time for the class made it difficult for me to pay attention.

29. The exams did not reflect the material we covered during class.

Brown SR (1980) Political subjectivity: applications of Q methodology in political science. Yale UniversityPress, New Haven

Page 11 of 1210.1007/s10956-008-9090-x



12/12

Brown SR (1999) Subjective behavior analysis. Unpublished manuscript. Read at a panel on Theobjective analysis of subjective behavior: William Stephensons Q methodology, annual convention ofthe Association for Behavior Analysis, Chicago. Retrieved March 18, 2005, fromhttp://facstaff.uww.edu/cottlec/QArchive/Aba99.htm

Cohen J (2003) Guidelines for establishing undergraduate bioinformatics courses. J Sci Educ Technol

12(4):449456

Jurczyk J, Ramlo S (2004, September). A new approach to performing course evaluations: Using Qmethodology to better understand student attitudes. Paper presented at the International Q MethodologyConference, Athens, GA

McKeown B (2001) Loss of meaning in likert scaling: a note on the Q methodological alternative. OpSubj 24:201206

McKeown B, Thomas D (1988) Q methodology. Sage Publications, Newbury Park, Calif

Ramlo S, McConnell D (2008) Perspectives of university faculty regarding faculty reading circles: a studyusing Q methodology. J Fac Dev 22(1):2532

Schmolck P (2002) PQMethod manual mirror. Unpublished manuscript. Retrieved April 29, 2004 fromhttp://www.rz.unibw-muenchen.de/~p41bsmk/qmethod/

Stephenson W (1955) The study of behavior: Q-technique and its methodology. University of ChicagoPress, Chicago

Valenta AL, Wigger U (1997) Q-methodology: definition and application in health care informatics. J AmMed Inform Assoc 4(6):501510

Page 12 of 1210.1007/s10956-008-9090-x

Eval an Inquiry-Based tics Course With Q OnLine JOST Version 2008

Documents

Transcript of Eval an Inquiry-Based tics Course With Q OnLine JOST Version 2008