ies.ed.gov

Connecting Research,Policy and Practice

Rebecca Kang McGill-Wilkinson, Ph.D.Institute of Education Sciences

Overview of Education Research & Development Centers Program

(84.305C)

ies.ed.gov 2

Outline

• Overview of IES• Research & Development Centers– Developmental Education Assessment & Instruction– Knowledge Utilization

• General requirements• Deadlines and notification

ies.ed.gov

Legislative Mission of IES

• Describe the condition and progress of education in the United States

• Identify education practices that improve academic achievement and access to education opportunities

• Evaluate the effectiveness of Federal and other education programs

3

ies.ed.gov 4

Organizational Structure of IES

4

National Board for Education

SciencesStandards &

Review Office

Office of the Director

National Center for Education Evaluation

National Center for Education Statistics

National Center for Education Research

National Center for

Special Education Research

ies.ed.gov

Missions of the Research Centers

• NCER– supports rigorous research that addresses the nation’s

most pressing education needs, from early childhood to adult education

• NCSER– sponsors a rigorous and comprehensive program of special

education research designed to expand the knowledge and understanding of infants, toddlers, and students with or at risk for disabilities from birth through high school

5

ies.ed.gov

IES Grant Programs: Research Objectives

• Develop or identify education interventions (practices, programs, policies, and approaches) – that enhance academic achievement– that can be widely deployed

• Identify what does not work and thereby encourage innovation and further research

• Understand the processes that underlie the effectiveness of education interventions and the variation in their effectiveness

6

ies.ed.gov

FY 2014 Research & Research Training Grant Programs

• Education Research Grants Program (84.305A)

• Education Research & Development Centers (84.305C)

• Statistical & Research Methodology in Education (84.305D)– Statistical & Research Methodology Grants – Early Career Statistical & Research Methodology Grants

• Partnerships & Collaborations Focused on Problems of Practice & Policy (84.305H)– Researcher-Practitioner Partnerships in Education Research– Continuous Improvement Research in Education– Evaluation of State & Local Education Programs & Policies

• Research Training Grant Programs in the Education Sciences (84.305B)– Predoctoral Interdisciplinary Research Training – Methods Training for Education Research– Training in Education Research Use & Practice

7

ies.ed.gov

NCER Investments by Competition

Program Number of Awards

2002-2012

Investment (in millions)2002-2012

Education Research 548 $897.2Research & Development Centers 20 $204.7Postdoctoral Research Training 35 $22Predoctoral Research Training 26 $121.8Evaluation of State & Local Programs & Policies 13 $53.4Statistical & Research Methodology in Education 33 $23.9Reading for Understanding 6 $113.4Preschool Curriculum Evaluation Research 12 $20.2Social & Character Development Research 7 $13.6Unsolicited 31 $44.9Small Business Innovation Research 58 $51

8

ies.ed.gov 9

Research & Development Centers: Purpose

• Contribute to solving U.S. education problems, improve the education system, and increase student achievement through– Research – Development – Evaluation – National leadership

ies.ed.gov 10

Research & Development Centers: Activities

• Focused research program– Contribute to solution of specific education problem within

the Center’s topic– Generate new knowledge and theories

• Supplemental studies– Research on supplemental questions within the Center’s

topic• Leadership and outreach– To researchers, practitioners, and policymakers– Advance the field and practice through evidence-based

research, practice, and policy

ies.ed.gov 11

Center Partnerships

• Institute encourages partnerships with stakeholder groups to increase– Relevance of work– Accessibility of publications– Usability of findings

ies.ed.gov 12

National Research & Development Center on Developmental Education Assessment

& Instruction

ies.ed.gov 13

Purpose

• Research– Document current developmental education practices– Identify promising programs– Support innovation– Evaluate programs that serve large numbers of students

and/or that can be expanded• Assess scalability of programs found to benefit students

• Leadership and outreach– Bring together policymakers, practitioners, and researchers – Assist states and postsecondary institutions to bring

effective models of developmental education to scale

ies.ed.gov 14

Background: Research

• Large numbers of students beginning postsecondary education – especially in community colleges and open-access 4-year institutions – are placed into developmental (remedial) reading, writing, or mathematics courses before they can accumulate credits– Many students are required to take multiple

developmental education courses

ies.ed.gov 15

Background: Research

• Research suggests tests and other practices used to place students in these courses may fail to predict who will benefit from developmental education

• Studies that track progress through coursework show mixed results– Students may not complete coursework– Students assigned to developmental education may be less

likely to graduate

ies.ed.gov 16

Background: Policy Concerns

• Decline in support for developmental education– In some corners, seen as more of a hindrance to students

than a support– Expenditures on developmental education

• Target for cost savings

• More students may be assigned to developmental education than need it

• Placement and instruction need to be revised or replaced

ies.ed.gov 17

Background: Ongoing Work to Improve Developmental Education

• Change placement procedures– Replacing standardized test with other measures

• E.g., high school transcript, college-readiness indicators– Change cut scores for tests

• Identify high school students’ need for developmental education and provide remediation – During high school rather than college– During a “bridge program” before college

• Providing additional supports during college– E.g., learning communities

ies.ed.gov 18

Background: Ongoing Work to Fundamentally Change Developmental Education

• Acceleration– Compress 1-year’s worth of material into a shorter period

• Modularization– Students complete a module of study at their own pace– Students who start with more knowledge can finish sooner

• Mainstreaming – Development education students take college-level courses

with additional support• Replacement– Replace conventional developmental math education

sequence with one on quantitative reasoning and statistics

ies.ed.gov 19

The Center’s Focused Program of Research

• One descriptive study– To document current developmental education practices

(assessment and instruction) in use by community colleges and open-access institutions

• Two evaluation studies– To determine if specific assessments or instructional

practices lead to improved student outcomes in reading, writing, math, English language proficiency, or any combination of these

ies.ed.gov 20

The Descriptive Study: Research Questions

• What are the assessment tools and practices used to assess reading, writing, and math skills of incoming students and place them in developmental education (versus college-level) courses?– Why do they use these tools and practices?

• What are the major strategies used to teach developmental education?

• Are different assessment and instructional approaches used for nontraditional students?

• Are the assessment and instructional approaches aligned with the Common Core State Standards?

• Applicants may expand or add additional research questions (and should provide justification for doing so)

ies.ed.gov 21

The Descriptive Study: Research Approach

• Institutional survey– Stronger if nationally representative of community colleges

and open-access institutions– If not national, stronger if focus on State or postsecondary

institution in which evaluation studies will take place• Qualitative interviews• Secondary data analysis• Combination of the above

ies.ed.gov 22

The Descriptive Study: Methodological Requirements

• Research questions• Population from which study sample will be selected• Measures• Data analysis plan• Timeline

ies.ed.gov 23

Evaluation Study 1: Assessment & Placement

• Possible Research Questions– How can institutions improve on current assessments and

placement practices?– What are the effects of alternative assessment and

placement strategies on students’ overall academic performance, persistence, and progress?

• You may propose expanded or modified research questions as long as they meet the purpose and requirements of the evaluation

ies.ed.gov 24

Evaluation Study 2: Instruction

• Possible Research Questions– How can institutions prepare students for college-level

courses in the shortest amount of time?– What are the impacts of different instructional approaches

on student learning, overall academic performance, persistence, and progress?

• You may propose expanded or modified research questions as long as they meet the purpose and requirements of the evaluation

ies.ed.gov 25

Evaluation Studies: Methodological Requirements

• Research questions• Sample and setting• Appropriate research design (meets WWC standards)• Power analysis• Measures– Proximal and distal outcomes

• Including student outcomes in reading, writing, math, English language proficiency, or some combination of these

– Fidelity of implementation– Comparison group practices

ies.ed.gov 26

Evaluation Studies: Methodological Requirements

• Moderators or mediators• Plan to analyze implementation fidelity and

comparison group practices• Data analysis plan– Overall impacts– Subgroup impacts (e.g., nontraditional students and

students taking more vs. fewer developmental ed courses)• Issues of scalability– Including cost-benefits of the programs evaluation

• Timeline

ies.ed.gov 27

Supplemental Studies

• May conduct smaller, supplemental research projects– Usually can be completed within 9 to 12 months– May include exploratory research, piloting innovations, or

further development of innovations• Center will work cooperatively with the Institute to

select and design supplemental studies• Must provide two examples of supplemental studies

in application– Short rationale and brief description of approach (2 to 3

paragraphs)

ies.ed.gov 28

Leadership & Outreach

• The Center will work with the Institute on its leadership and outreach activities. These may include:– Convening policymakers, practitioners, and researchers– Assisting efforts by States, colleges, and universities to

bring effective models to scale• The Institute encourages a Center website, webinars,

podcasts, and other innovative uses of technology• You should describe at least two examples of

activities in application

ies.ed.gov 29

Significance of Center

• Significance section at start of Research Narrative• Discuss the significance of your proposed Center– Your vision of the Center – What problems it is intended to address– How your set of research and leadership activities will

work together to help students in developmental ed– How the research will be relevant to community colleges

and open-access 4-year institutions• Describe active partnerships with States or

postsecondary systems and their role in the work

ies.ed.gov 30

Maximum Awards

• 5 years• $10,000,000• Maximum of one award• Focused program of research must comprise at least

75% of the total budget

ies.ed.gov 31

National Research & Development Center on Knowledge Utilization

ies.ed.gov 32

Purpose

• Research– Develop tools for observing and measuring research use in

schools– Identify conditions under which practitioners use research

and factors that promote or inhibit research use in schools– Identify or evaluate strategies that researchers can adopt

to make their work more meaningful to and impactful on education practice

• Leadership and outreach– Effective means of knowledge transfer to improve school

performance and student outcomes

ies.ed.gov 33

Background

• Accumulation of high-quality research and evaluation on impact of education policies and interventions

• Concern that this work is not used by practitioners– To adopt interventions with beneficial effects on students– To discard interventions with null or adverse effects

ies.ed.gov 34

Background: Potential Causes

• Misalignment between research questions (e.g., “Does it work?”) and practitioners’ concerns (e.g., “How do I use it with my students?”)

• Misunderstanding of social context of practitioners– Research use is multi-level phenomenon

• Need new ways of working with practitioners– Partnerships– Bidirectional model of relationship– Intermediary organizations

ies.ed.gov 35

The Center’s Focused Program of Research

• One measurement study– Develop and validate tools for observing and measuring

research use in schools and school districts• Two descriptive studies, or one descriptive study +

one evaluation/pilot study– Identify the conditions under which research is used and

factors that promote research/inhibit research use in schools and districts

– Identify or evaluate the skills and strategies researchers can adopt to make their work more meaningful and have a greater impact on practice

ies.ed.gov 36

The Measurement Study: Research Questions

• What are the most reliable and valid indicators of school staff using education research in their decision making?

• What are the most reliable and valid approaches to capture variation in research use across schools, school districts, and over time?

• Applicants may expand or add additional research questions (and should provide justification for doing so)

ies.ed.gov 37

The Measurement Study: Methodological Requirements

• Conceptual framework to provide basis for assessment, intended population, and validation

• Sample and setting• Characteristics, size, analytic adequacy of sample• Methods for developing and validating• Methods for gathering psychometric evidence• Appropriateness for end users• Need for adaptation for different contexts and purposes• Timeline

ies.ed.gov 38

Descriptive Study 1: Research Questions

• What role do State and local education agencies play in making research available to school administrators, faculty, and staff?

• What role do school leaders play?• How do education agencies, school leaders, and teachers obtain

and process scientific information?• How do social networks within schools and school districts appear

to support or hinder research use?• What are the major incentives or disincentives for practitioners to

better understand and apply education research?• What contextual factors seem to affect the use of research in

schools and school districts?• Applicants may expand or add additional research questions (and

should provide justification for doing so)

ies.ed.gov 39

Descriptive Study 2: Research Questions

• Are partnerships between researchers and practitioners associated with greater use?

• Are intermediaries associated with greater use?• What communication strategies or technological

innovations are associated with greater use?• Applicants may expand or add additional research

questions (and should provide justification for doing so)

ies.ed.gov 40

Optional Evaluation or Pilot Study

In lieu of second descriptive study• Examine efficacy of researcher practices on research

useOR

• Rigorous evaluation of impact of particular practiceOR

• Pilot study to show promise of particular practice

ies.ed.gov 41

Descriptive Studies: Methodological Requirements

• Research questions• Population from which study sample will be selected– Institute encourages you to select two or more research

sites• Measures• Data analysis plan• Timeline

ies.ed.gov 42

Optional Evaluation/Pilot Study: Methodological Requirements

• Research questions• Sample and setting• Appropriate research design– if doing an evaluation, the design should meet WWC

standards• Power analysis• Measures– Proximal and distal outcomes– Fidelity of implementation– Comparison group practices

ies.ed.gov 43

Optional Evaluation/Pilot Study: Methodological Requirements

• Moderators or mediators• Plan to analyze implementation fidelity and

comparison group practices• Data analysis plan• Timeline

ies.ed.gov 44

Leadership & Outreach

• Innovative approaches to fostering dialogue among practitioners and researchers

• May take advantage of annual conferences• Institute encourages websites• Institute is open to learning how other applied social

sciences encourage research use among practitioners• Describe at least two examples of activities in

application

ies.ed.gov 45

Supplemental Studies

• May conduct smaller, supplemental research projects– Usually can be completed within 9 to 12 months

• Center will work cooperatively with the Institute to select and design supplemental studies

• Must provide two examples of supplemental studies in application– Short rationale and brief description of approach (2 to 3

paragraphs)

ies.ed.gov 46

Significance of Center

• Significance section at start of Research Narrative• Discuss the significance of your proposed Center– Your vision of the Center – What problems it is intended to address– How your set of research and leadership activities will

work together to promote the use of scientific education research in schools and districts

• Describe how you will engage researchers and practitioners in the work and communication of findings

ies.ed.gov 47

Maximum Awards

• 5 years• $5,000,000• Maximum of one award• Focused program of research must comprise at least

75% of the total budget

ies.ed.gov 48

General Requirements

ies.ed.gov 49

Additional Sections of the Research Narrative for Both Centers

• In the 35-page Research Narrative, discuss…– Significance– Research Plan for the Focused Program of Research

(including supplemental studies)– Other Center Activities (Leadership and Outreach)– Management and Institutional Resources– Personnel

• We’ve discussed the first three and now let’s move to the last two

ies.ed.gov 50

General Requirements: Management & Institutional Resources

• Describe plans and procedures for overall management of the Center and its activities

• If plan is to work in schools or other education delivery settings in first year, must include letter(s) of agreement from education organization(s)

• If developer/distributer of program or intervention is part of Center, describe role and how objectivity will be maintained

ies.ed.gov 51

General Requirements: Personnel

• Expertise in content area(s)• Methodological expertise• Sufficient experience working with education

delivery settings• Experience relevant to national leadership– Explain why proposed Center staff is qualified to fulfill

leadership role

ies.ed.gov

Application Deadline

Application Package Posted

Start Dates

Sept 4, 20134:30:00 PM

DC Time

June 6, 2013 July 1, 2014to

Sept 1, 2014

Important Dates & Deadlines

52

ies.ed.gov

Notification

• All applicants will receive e-mail notification that the following information is available via the Applicant Notification System (ANS):• Status of award• Reviewer summary statement

• If you are not granted an award the first time, consider resubmitting and talking with your program officer

ies.ed.gov 54

Questions?

Becky McGill-Wilkinson rebecca.mcgill@ed.gov