Evaluation Methods and Findings of the Field-tested Learning Assessment Guide
description
Transcript of Evaluation Methods and Findings of the Field-tested Learning Assessment Guide
![Page 1: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/1.jpg)
Evaluation Methods and Findings of the Field-tested
Learning Assessment Guide
Sue Daffinrud
Olga Lucia Herrera
![Page 2: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/2.jpg)
The Motivation for the FLAG
Motivated by needs of faculty who wanted centralized place to learn about already-verified assessment techniques and tools (Third Annual NISE Forum)
Does the FLAG satisfy part of this need?
![Page 3: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/3.jpg)
FLAG Structure
Five major sections Getting Started Matching Goals to CATs CATs Searchable Tools Resources
![Page 4: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/4.jpg)
FLAG target audience
Instructors who:
teach freshman and sophomore-level science, math, and engineering courses
and are interested in learning about new
approaches to assessment
![Page 5: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/5.jpg)
FLAG Evaluation
Does the FLAG website fulfill some of the assessment needs of its target audience?
Formative and Summative Evaluation
![Page 6: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/6.jpg)
Identifying target audience sample
Selected subset of SME disciplines Biology, chemistry, engineering, mathematics,
and physics Selected range of institutions
By Carnegie classification Associate of Arts, Baccalaureate, Masters,
Doctoral, and Research Region of country
![Page 7: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/7.jpg)
Identifying target audience sample
Selected faculty through their participation in some initiative which has student learning as its focus. bioQUEST, Worskhop Biology, OCEPT Chemistry Coalitions Engineering Coalitions, EESP Project NExT Workshop Physics
![Page 8: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/8.jpg)
50 faculty who have demonstrated an interest in “learning, not teaching”
Bio Chem Engr Math Phys
Assoc 2 2 2 2 2
Bacc 2 2 2 2 2
Masters 2 2 2 2 2
Doctoral 2 2 2 2 2
Research 2 2 2 2 2
![Page 9: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/9.jpg)
Contacting participants
Called and emailed faculty requesting their participation
Participation involved: Browsing through FLAG for 30 minutes Providing feedback in a 30 minute interview
High participation rate
![Page 10: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/10.jpg)
Interviews
Participants were asked about: Their current teaching practice How they learn about new
assessment/teaching techniques Impressions of FLAG, both specific and
general Whether would recommend FLAG to others
![Page 11: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/11.jpg)
Data Gathering Status
Have interviewed 32 faculty Will interview the remaining 18 in
December
![Page 12: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/12.jpg)
Preliminary findings
Demographics of respondents
Technical issues
Themes of Use and Need
Dissemination
![Page 13: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/13.jpg)
Demographics
Range of teaching experience (2-22 yrs) Critical of the value of lecture Goals: improving students’ interest in science Role: facilitator for their learning Info on assessment gained through
conversations with fellow faculty, conferences in their discipline
Few “surf” the web
![Page 14: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/14.jpg)
Technical Issues – Navigation patterns
Participants spent between 30 minutes to 4 hours in the FLAG, most about an hour.
Many moved “left to right”
![Page 15: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/15.jpg)
Technical Issues
Almost all found the web site Easy to read Structured well Could find what they needed Enjoyed being able to download documents
Suggestions Provide more obvious description of site structure Provide better navigation instructions
![Page 16: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/16.jpg)
Themes of Use/Need - Overall
Overall response: Positive
Almost all respondents had already or had plans to recommend the site to other faculty, faculty developers, or other groups
![Page 17: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/17.jpg)
Themes of Use/Need - Sections
Has teaching component Matching Goals to CATs Primer
Availability of links to other web sites Annotated bibliography References
“Empty” sections create expectation Searchable tools “Experts” by state
![Page 18: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/18.jpg)
Themes of Use/Need - CATs
Time is a factor in browsing and in applicability First page of CATs (why, how, what’s
involved) People selected:
CATs which were familiar to them CATs which best fit their goals and time
![Page 19: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/19.jpg)
Themes of Use/Need – CATs (cont)
People want a complete picture Like the case study approach When and how to use the CAT What to do with the information How to get students “on board” Clear and useful examples
People want “tools” from their discipline to varying degrees
![Page 20: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/20.jpg)
Dissemination of FLAG
Evaluation AS dissemination Hope to find out how best to reach people
like our sample Discipline-specific professional meetings Workshops People on campus who do assessment Email to appropriate email lists
![Page 21: Evaluation Methods and Findings of the Field-tested Learning Assessment Guide](https://reader034.fdocuments.in/reader034/viewer/2022042822/56814e1b550346895dbb8119/html5/thumbnails/21.jpg)
Final Notes
Submitted abstract for presentation at AAHE 2000
Write monograph or paper Provide more formative information
to Tony To ILT group as they develop site