Draft of the

Conceptual Framework for Evaluation & Assessment of the

National Science Foundation (NSF)

Alliance for Graduate Education & the Professoriate (AGEP) Program

Developed by

Yolanda S. George, AAAS Education & Human Resources Programs and

Patricia Campbell, Campbell-Kibler Associates, Inc.

With input from the AGEP Community

Help AGEP awardees to build a comprehensive evaluation and

assessment infrastructure to examine their graduate education enterprise in terms of creating a diverse S&E PhD

workforce.

The primary goal of the AGEP Program evaluation capacity building project is to:

Recruitment and talent spotting strategies. Admissions and selection processes.Financial aid, particularly as it affects new graduate student enrollment and retention.Academic preparation & support programs, particularly during the early course-taking years.Advising, mentoring, and retention. Workforce preparation.PhD career counseling and employment placement.

The evaluation framework and model for change are focused on graduate student affairs, including:

Graduate School and Academic Deans Provosts and Other Administrators

S&E Department Chairs

S&E Departmental Admissions and Selection Committees

S&E Departmental Graduate Program Directors or Advisors

S&E Faculty

Key Administrators Using Evaluation & Assessment to Make informed Decisions

Race/Ethnicity

Gender

Citizenship

S&E Departments

Collection And Use Of Disaggregated Data

UM applicants & admits

New or first time UM enrollees;

Overall UM enrollment;

UM students advancing to doctoral candidacy;

UM students completing degrees (Master’s or PhD).

Post PhD Employment

Quantitative data should be used to examine patterns and trends in number and percentage of:

Number and percentage of S&E departments involved Number of S&E faculty involved Number of S&E faculty involved in graduate student recruitment.S&E departmental practices related to graduate student admissions and selection.S&E departmental practices related to academic preparation, advising and workforce mentoringS&E departmental practices related to faculty evaluation and incentives.Family leave policies and practices within the graduate school or departments. As related to graduate students.Data collection practices, particularly use of disaggregated data, at the in decision-making.

Indicators of Infrastructure Changes in Graduate Programs

The quality of the implementation including participants’ and/or external reviewers’ perceptions about the intended objectives of a workshop or program component.

Time-on-task

The effectiveness of the interventions or to what extent did the intervention achieve its intended objectives.

Also, data and information need to be collected on

Quantitative and qualitative studies about policies, practices, and programs in the Graduate School & S&E departments should be used to determine

What is working and not working

What works for whom.

Using the meeting to begin to reflect on infrastructure changes in graduate schools

How building a comprehensive evaluation and assessment infrastructure

How to better work with institutional researchers, other campus research units, external evaluator

How to involve S & E departments

AGEP Program Director will review Expectations

NSF HRD Evaluation Office & Performance Assessment

Lessons Learned by Some AGEP Institutions

Strand 1 --Building a data collection infrastructure related to graduate students

and their progression to the PhD.

CGS PhD Completion Project

Working with STEM departments and faculty to ensure retention in PhD programs, particularly during the first years of graduate school

Strategies for helping students to be successful on qualifying exams

Strategies to reduce time to degree

Strand 2Retention & Progress to the PhD

Strand 3

Assessing recruitment and departmental admissions/selections efforts.

Other Agenda Elements

Foundation & Discipline Perspectives

NSF Director

Using WebCASPAR

Update on AGEP Data Collection

NSF Director

This is a community effort.

The framework is evolving

This is not an easy journey, neither is it one that can be completed quickly

In Summary