Issues Surrounding the Evaluation of Teacher Internship

Programs

Donna Barrett

Center for Education Integrating Science, Mathematics & Computing

Georgia Institute of TechnologyAtlanta, Georgia

…. a collaborative effort of corporations, universities and school districts designed to enhance mathematics and science experiences of Georgia teachers and their students. Since 1991, teachers have been placed in over 1100 GIFT internships throughout the state.

Goals of

Increasing the teacher’s personal knowledge base of science, mathematics and technology

Enhancing the teacher’s enthusiasm for teaching

Providing access to new techniques and procedures

Encouraging partnerships with business and education

The GIFT Experience - Commonalities Common application Internship in STEM-based work setting Emphasis on “Real World” problem solving and

reasoning Development of Inquiry Based Unit – Action Plan

The GIFT Experience- Variables

Location of placement University Corporate Informal science/public education

Goal of placement ”Traditional” Student research International Curriculum development

Length of placement: 4-8 weeks

The GIFT Experience - Variables Geography

Placements across Georgia Teaching Level

High school vs. middle school Teaching Field

Science, math, technology, social science Type of school

affluent vs. low income, stable vs. changing, minority vs. majority

GIFT Evaluation Plan An Advisory Board composed of 25 members

from corporations, universities, and school districts

Utilizes the strategy of subcommittee phone conferences to provide guidance on program issues such as evaluation

Evaluation subcommittee has provided input on program evaluation

Worked with an internal evaluator to develop a GIFT Logic Model and develop assessments

Assessing Impact on Teachers Participant Reactions

Numbers Are participants satisfied? Teacher and Mentor Surveys

Participant Learning Action Plan Analysis What have teachers learned about inquiry based

learning?

GIFT Logic Model adapted from Guskey’s 5 Level Model for Assessing Professional Development

Assessing Impact on Teachers Participant Actions

How have they applied inquiry based learning in their classroom?

Surveys – Pre-Summer, Post-Summer & School Year Follow-up (Post-GIFT)

Alumni Survey Classroom Observation

GIFT Logic Model adapted from Guskey’s 5 Level Model for Assessing Professional Development

Using inquiry and hand-on activities

Pre-GIFTN=47

Post-GIFTN=24

Minor23%

Moderate43%

Major34%

Minor0% Moderate

21%

Major79%

Pre and Post Surveys

Based on surveys from: www.retnetwork.org

Assessing Impact on Teachers Organizational Change

Change in school infrastructure Teachers as agents of change Alumni Survey Focus Group Discussions Classroom Observation

GIFT Logic Model adapted from Guskey’s 5 Level Model for Assessing Professional Development

GIFT Alumni Survey Searching for 668 former participants GIFT database/Districts/Websites/Searches 123 Fellows – No contact information 545 Contact Attempts: 382 E-mail & 163

mailed; 36 E-mail and/or mail returned

509 Total Sample Size 127 - Surveys returned Return Rate: 25%

To what extent, if any, do you feel that you experienced each of the following types of learning as a result of your participation in GIFT? Strongly Agree

or Agree

I gained a greater understanding of the applications science, mathematics, and/or technology in every day life. 93.10%

I gained greater understanding of fundamental concepts in science, mathematics or technology. 84.30%

I increased my knowledge of current issues in scientific or mathematical research. 91.20%

I increased my knowledge of careers that utilize science, mathematics, and/or technology. 95.10%

I gained an appreciation of the difficulties some students encounter when learning new material. 78.50%

It increased my comfort level with inquiry-based learning strategies. 76.00%

It increased my ability to incorporate "real life" examples of the subjects I teach. 96.10%

Leadership Roles16% reported they became a department or grade level chair person AFTER their GIFT experience

21% reported they became a school district leader (Principal, AP or Coordinator) AFTER their GIFT experience

Impact of GIFT on Involvement with Students

68%

30%

60%

30%

65%

14%

10%

10%

13%

6%

15%

11%

10%

17%

16%

4%

50%

20%

40%

13%

0%

20%

40%

60%

80%

100%

Sponsoreda ScienceOlympiad

Team

Long TermScienceProjects

ScienceProjectsplaced at

the Districtor State

Sponsoreda Club

related toSTEM

Involved inSummerScience

Programsfor Students

No Prior After Before & After

Continuing Contact with Mentor

24%35%

70%76% 81%

91%44%

39%

24%21% 12%

7%

18%15%

4% 3%5% 0%

15% 11%2% 0% 2% 2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Maintainedcontact

follow ingGIFT

experience.

Maintainedcontact via

e-mail

Mentorvisited

classroom

Studentsvisitedmentor

Mentorassisted

w ith studentprojects

Published ascientif icpaper/

presentedtogether

Not at all Year Following 2-3 years following More than 3 years following

Limitations of Data

The data is self reported, although anonymous

Return rate of Alumni Survey – 25% Sample sizes often vary on the pre and post

surveys Focus Groups have provided more specific

information

Challenges in Correlating Student Achievement with Teacher Professional Development Designing a scientifically valid study with

appropriate controls Appropriate measures of student

achievement across grade levels, subjects Access to student achievement data Teachers changing schools, subjects and

grade levels

Assessing the Impact on Students Student Achievement

Connecting the dots through a literature review Encouraging the use Research based best

practices (Inquiry, Problem Based Learning, etc.) Impacting teacher content knowledge

GIFT Logic Model adapted from Guskey’s 5 Level Model for Assessing Professional Development

Future Evaluation Goals

Collect student achievement data of GIFT participants – proposed study with a local school district to collect preliminary data Criterion-Referenced Competency Tests (CRCT)

– Middle Grades End of Course Test (EOCT )– High School

Biology, Physical Science, Algebra, Geometry Aggregate data from specific class and compare

to state average Pre GIFT data, Post GIFT data, YR 1, YR 2 Post

Evaluation Tools/Resources Columbia Program: http://scienceteacherprogram

.org/ Industry Initiatives for Science and Math Education

(IISME): www.iisme.org RET Network Surveys: www.retnetwork.org SWEPT study: www.sweptstudy.org TRE Conference on Research Experiences. http:

//omp.gso.uri.edu/CTRE/ Visiting a High School Inquiry Classroom: How to

Prepare and Observe: http://cse.edc.org/pdfs/products/observerguide.pdf

For more information:

Donna BarrettProgram Director

donna.barrett@ceismc.gatech.edu

(404) 894.7530

www.ceismc.gatech.edu/gift