Health Promotion and Family Engagement Initiatives in Race to the Top-Early Learning Challenge

An Implementation Evaluation

Karista Gallick, MPP 5/22/2015

Prepared for the Administration for Children and Families

and PUAF790 Project Course, Maryland School of Public Policy

Executive Summary

Race to the Top-Early Learning Challenge (RTT-ELC) started in 2011 as a competitive grant

program for states to implement high-quality early learning programs and increase these programs’

infrastructure to improve school readiness. As part of the program, states can choose how they want to

address health promotion and family engagement. Children’s health and wellbeing allow them to perform

better in school, and caregivers’ awareness of health or development issues early on allow children to

receive care if necessary. Additionally, caregivers’ early awareness of the importance of health sets

children on lifelong paths of healthy habits and preventative care. Family engagement strengthens

parents’ involvement in their children’s education, health, and wellbeing. Family engagement components

in early education settings can initiate the process of parental involvement early on in children’s lives in

order to create lifelong habits of parental involvement. Several components exist to RTT-ELC, but these

two components of health promotion and family engagement serve as the subject of this implementation

evaluation.

This implementation evaluation first outlines RTT-ELC as a whole and shows where child health

and family engagement fit into the rest of the program. After introducing RTT-ELC, I review the

literature which discusses the importance of health promotion and family engagement as part of a child’s

early education. Following the literature review, I evaluate states’ implementations of their respective

health and family engagement initiatives and determine to what extent these implementations match

states’ initial program designs. The purpose of this evaluation is to see if states’ use the RTT-ELC grant

funds in the ways they specify and to provide a broader picture of what RTT-ELC states are doing with

health and family engagement. ACF is interested in monitoring trends in health promotion and family

engagement across grantee states; this broad picture will allow ACF to see how the rollout of RTT-ELC is

proceeding and what challenges states face in their implementations. Findings indicate mixed successes in

implementations across states, but many states share some common experiences, and I conclude with

recommendations based on these findings; these recommendations could stand to benefit both grantee

states and ACF.

1

Introduction

Race to the Top-Early Learning Challenge (RTT-ELC), jointly administered by the

Administration for Children and Families (ACF) (within the Department of Health and Human Services)

and the Department of Education (ED), started in 2011 as a competitive grant program for states to

implement and expand their high-quality early learning programs. States have four years to implement

their grants and they submit annual performance reports to both ACF and ED. As of December 2014, the

federal government has allocated nearly $1 billion to RTT-ELC since the program’s inception. The

Obama Administration highly prioritizes early childhood education. In his 2015 State of the Union

address, President Obama called for increased investments in early childhood education. The secretaries

of the agencies which co-administer RTT-ELC also recognize the importance of quality early education.

According to Department of Education Secretary Arne Duncan, “[h]igh-quality early education can be a

game-changer for the children and families who need the most support” and Health and Human Services

(HHS) Secretary Sylvia Burwell remarked, “[w]hen we invest in early education, the benefits can last a

lifetime. We all gain when our country has a stronger, more productive workforce, lower crime rates, and

less need for public assistance” (U.S. Department of Education and U.S. Department of Health and

Human Services, 2014).

The format of RTT-ELC allows grantee states to build high quality early childhood education

programs through foundational areas as well as some options to expand upon those foundations. The first

two components of RTT-ELC, called Core Areas, call for states to implement “Successful State Systems”

and “High Quality Accountable Programs,” respectively. These components serve as the backbone to the

rest of a state’s early childhood education program; these qualities must be in place in order for further

developments to take root and thrive.

“Successful State Systems” requires states to outline their early childhood education system as a

whole. They must support their states’ current early childhood education landscapes and infrastructure

(i.e., policies, legislation, and partnerships) with RTT-ELC. In this section states spell out how they work

with partners across sectors in developing their early childhood education system and how they ensure all

2

involved partners have the same vision and goals in implementing RTT-ELC. RTT-ELC intends to bring

together states’ patchworks of previously uncoordinated early learning programs and form a more

coherent birth-5 learning system, so states must demonstrate their intentions to streamline early learning

programs and related services. This section comes first since it provides the framework for the rest of

RTT-ELC initiatives.

The second core area “High Quality Accountable Programs,” is also a requirement for states. As

part of this section, states must develop or improve a Tiered Quality Rating and Improvement System

(TQRIS) in order to rate and track the quality of early childhood education programs in the state. Prior to

RTT-ELC, several states had some version of a Quality Ratings System, but these were often developed

with varying criteria without similar benchmarks. RTT-ELC gives states opportunities to standardize their

systems, bring more early learning programs into the system, target programs for improvement, and keep

programs accountable. When choosing a program for their children, parents can use the TQRIS as a

metric to determine what program would best suit their children.

The next three components, called “Focus Investment Areas,” (FIAs) allow states to choose how

they want to address certain aspects of early childhood learning in their respective states. FIA C is a broad

category with the overarching theme of “Promoting Early Learning and Development Outcomes for

Children” (U.S. Department of Education and U.S. Department of Health and Human Services, 2014).

Focus Investment Area C gives states options on how to promote child development outcomes. Within

FIA C are four criteria. States can select from these criteria ways to further improve the quality of their

early childhood education systems:

(C)(1) Developing and using statewide, high-quality Early Learning and Development Standards

(C)(2) Supporting effective uses of Comprehensive Assessment Systems

(C)(3) Identifying and addressing the health, behavioral, and developmental needs of Children with High Needs1 to improve school readiness

1The Notice to Applicants defines “Children with High Needs” as “children from birth through kindergarten entry who are from Low-Income families or otherwise in need of special assistance and support, including children who

3

(C)(4) Engaging and supporting families (U.S. Department of Education and U.S. Department of Health and Human Services, 2014a, 6)

(C)(1) allows states to create Early Learning and Development Standards, which spell out what

children should be able to know and do at certain ages. Another option under FIA C, (C)(2), allows states

to supporting effective uses of Comprehensive Assessment Systems to: ensure that assessment

approaches and instruments are appropriate children; avoid duplication of assessments; train Early

Childhood Educators to appropriately administer assessments and interpret assessment data to inform and

improve classroom instruction; and to share assessment data and results with parents. (C)(3) and (C)(4)

allow states to address health promotion and family engagement in order to ensure children’s wellbeing

and engage families in keeping children healthy and ready to learn. (C)(3) and ( C)(4) each have their

own sub-criteria to which states can write. Figures 1 and 2 detail these sub-criteria.

FIA D “Supporting and Strengthening the Early Childhood Workforce,” gives states the

opportunity to strengthen their early childhood education systems’ workforce. As part of FIA D, states

can develop Workforce Knowledge and Competency Frameworks, which spells out what members of the

early childhood workforce need to know and be able to do. States can also tie these frameworks to a

progression of credentials or support and improve the knowledge, skills, and abilities of early childhood

educators through various methods, including professional development, scholarships, or the creation of

certificate programs in partnership with local colleges and universities. Strengthening the early childhood

education workforce builds off of FIA C in that trained teachers can help identify child health and

development issues and work with parents to address these issues and engage parents in their children’s

education and development.

Under FIA E, states can choose to address Kindergarten Entry Assessments (KEAs), which

determine the effectiveness of early childhood education programs. KEA results can reveal strengths and

gaps in early learning programs, and stakeholders can use these results to alter early learning programs

accordingly in order to best prepare children for kindergarten. FIA E also allows states to work on

have disabilities or developmental delays; who are English learners; who reside on ‘Indian lands;’… who are migrant, homeless, or in foster care; and other children as identified by the State” (Federal Register, 2013, 53995).

4

building, coordinating, or improving their early childhood data systems. With more streamlined and

accessible data, early childhood education stakeholders can more easily determine programs’ successes

and determine how to make improvements to programs and better meet children’s needs.

All of the Core and Focused Investment Areas sections together work to form higher-quality early

childhood education systems in grantee states. Core Areas A and B allow states to develop the

foundational components early childhood education programs and systems, while FIAs C, D, and E allow

states to build quality and address several of the facets that make up early childhood education systems.

This paper focuses on the health and family engagement sections of FIA C. The section below describes

the extent to which states have implemented health and family engagement improvements in RTT-ELC.

Health and Family Engagement in RTT-ELC

While high quality early education can help all children prepare for kindergarten, the effects of

early learning programs are especially profound among children in low-income households or with other

special needs (some literature on this subject is discussed further in the next section). However, low-

income children do not always have the opportunities to participate in these programs, and RTT-ELC

works to provide quality early learning opportunities for such children. However, an early learning setting

on its own will not suffice in helping to prepare children for the start of school as well as the rest of their

academic careers; children and early learning programs do not exist in a vacuum. Factors external to early

childhood education centers, such as children’s family and home lives, affect children’s development and

abilities to learn. Therefore, in addition to increasing the prevalence, quality, and accountability of early

education programs, part of RTT-ELC also consists of addressing health promotion and family

engagement. ACF and ED recognize that preventative child health and active family engagement both

play a role in children’s lives and, along with a quality early education program, can create positive

outcomes for young children. Since there are many strategies to approach health and family engagement,

ACF and ED gave grantee states opportunities to choose how to address child health and family

engagement in their early childhood education programs.

5

FIAs C(3) and C(4), health and family engagement, respectively, are the subjects of this

implementation evaluation. Each criterion has further specified sub-criteria, and states can choose

to address as many of these sub-criteria as they choose. Figure 1 lists the sub-criterion states can

address. C(3)(d) contains four performance measures which ACF asks states to address if they

chose to address health promotion.

Figure 1

Sub-Criteria of C(3): Health Promotion

a. Establishing a progression of standards for ensuring children’s health and safety; ensuring that health and behavioral screening and follow-up occur; and promoting children’s physical, social, and emotional development across the levels of its Program Standards

b. Increasing the number of Early Childhood Educators who are trained and supported on an on-going basis in meeting the health standards

c. Promoting healthy eating habits, improving nutrition, expanding physical activity, and

d. Leveraging existing resources to meet ambitious yet achievable annual targets to increase the number of Children with High Needs who 1. Are Screened using Screening Measures that align with the

Medicaid Early Periodic Screening, Diagnostic, and Treatment benefit of the Social Security Act or the well-baby and well-child services available through the Children’s Health insurance program, and that, as appropriate, are consistent with the Child Find provisions in IDEA

2. Are referred for services based on the results of those screenings, and where appropriate, received follow up; and

3. Participate in ongoing health care as part of a schedule of well-child care, including the number of children who are up to date in a schedule of well-child care

e. Developing a comprehensive approach to increase the capacity and improve the overall quality of Early Learning and Development Programs to support and address the social and emotional development (including infant-early childhood mental health) of children from birth to age five*

*this sub-criterion starts started in Phase III

Source: The Federal Register, Applications for New Awards; Race to the Top – Early Learning Challenge, 2011. *this sub-criterion starts started in Phase III

6

C(4) also has sub-criteria, from which states can choose to target their family engagement

strategies; figure 2 lists these sub-criteria.

Family engagement is a nebulous concept and carries different meanings for different parties.

ACF uses the Office of Head Start’s definition of family engagement (since Head Start is also under ACF

and provides a comprehensive definition). According to Head Start’s Parent, Family, and Engagement

Community Framework, parent and family engagement “is about building relationships with families that

support well-being, strong relationships between parents and their children, and ongoing learning and

development for both parents and children” (The Office of Head Start, 2011, 1). Engagement, in this

sense, can refer to an outcome, goal, set of strategies, or relationships between teachers, parents, and

schools. Engagement is a continuous process and can change as children grow; therefore, ACF seeks to

build foundation for successful parent and family engagement and create “engagement pathways” through

Figure 2

Sub-Criteria of C(4): Family Engagement

a. Establishing a progression of culturally and linguistically

appropriate standards for family engagement across the levels of

its Program Standards, including activities that enhance the

capacity of families to support their children’s education and

development;

b. Increasing the number and percentage of Early Childhood

Educators trained and supported on an on-going basis to

implement the family engagement strategies included in the

Program Standards; and

c. Promoting family support and engagement statewide, including

by leveraging other existing resources such as through home

visiting programs, other family-serving agencies, and through

outreach to family, friend, and neighbor caregivers.

Source: The Federal Register, Applications for New Awards; Race to the Top – Early Learning Challenge, 2011.

7

which engagement can develop. ACF recognizes that engagement is a continuous process and can change

as children grow; therefore, ACF encourages states to build a foundation for successful parent and family

engagement and create “engagement pathways” through which engagement can develop.

The RTT-ELC Notice Inviting Applicants in the Federal Register (grant application for RTT-

ELC), which invited states to apply for and participate in RTT-ELC, provided several examples of family

engagement activities as well:

[p]arent access to the program, ongoing two-way communication with families, parent

education in child development, outreach to fathers and other family members, training

and support for families as children move to preschool and kindergarten, social networks

of support, intergenerational activities, linkages with community supports and adult and

family literacy programs, parent involvement in decision making… parent leadership

development [and] activities that enhance the capacity of family to support their

children’s education and development (Department of Education and Department of

Health and Human Services, 2013, 54005).

These examples are not an exhaustive list; the Notice to Applicants specified that states could

choose other programs and activities that pertain to family engagement. Applications, though,

were judged on how well states programs and activities addressed the FIA and sub-criteria which

ACF specified.

Brief Review of the Literature

Extensive research points to the impacts of early childhood education programs with

elements of health and family engagement on positive outcomes in cognitive development and

learning in children. Positive social and economic outcomes manifest in adult years as well. Some

of the most renowned studies, even though several years, or even decades, continue to influence

debates surrounding early childhood education.

The High/Scope Perry Preschool Study has gained renown in early childhood education

research for its experimental and longitudinal design. Schweinhart and Weikart (1980) tracked

low-income African American 3 and 4 year-olds who participated in the program in Ypsilanti,

8

MI, in 1962 (along with the control group) for 11 years following the program’s conclusion, and

other researchers have studied these participants even further into their adult years. The Perry

Preschool Program included parental involvement and preventative health programing via home

visits that included the mother and child together. Providers visited children at home along with

their mothers to ensure children’s wellbeing at home and to demonstrate the importance of the

role of parents in early child development. These visits lasted 1.5 hours per week and

additionally, children in the program group attended a group preschool program 12.5 hours per

week. This schedule lasted 30 weeks of the year for one year for the first cohort of children, and

two years for the following two cohorts.

Schweinhart and Weikart (1980) found a significant increase in program participants’

cognitive abilities in preschool through first grade, after which the effects dwindled (31-32).

Schweinhart and Weikart (1980) also found higher levels of academic achievement (as measured

by the percent of items passed) during elementary school and middle schools among the Perry

Preschool Participants than the control group. These differences were noticeable at age 14 and

significant in language achievement and grammar (id., 37-43). Schweinhart andWeikart (1980)

suggest declines in achievement, especially after first grade, result from the lack of continued

support after the program’s conclusion (86-87). Structured support in their early years led to

short-term high gains, but once children continued their academic careers, teachers’ and

caregivers’ commitments to achievement did not match the commitment of the Perry staff who

strove “to offer the best possible program to each child enrolled” (id.). Without adults’ continuing

commitments and motivational support to children throughout their academic lives, in turn,

children’s own commitments to school reflect adults’ expectations and attitudes. Schweinhart and

Weikart (1980) argue for “invest[ing]… more fully in…institutions,” such as “family, schools,

[and] the government” (id). The opposite school of thought regarding the results of Perry

Preschool argues the declines in achievement indicate the program did not succeed. Olsen (1999)

9

points out while the program group in the study outperformed the control group, those in the

program group, “they still fared poorly compared with mainstream children” (Olsen, 1999, 13).

Schweinhart and Weikart (1993) studied the Perry Preschool cohort in years following

secondary school as well. Their study of the cohort at age 27 revealed of former program

participants, 71% had academic achievement beyond the 12th grade, as opposed to 54% of the

control group. Program participants were also more likely than their control group peers to

demonstrate other positive indicators. Twenty nine percent of Perry Preschool Program

participants made over $2,000, as opposed to 7% of the control group, 36% of participants owned

a home as opposed to 13% of the control group, and 59% had received welfare as an adult,

whereas 80% of the control group had received it (54).

Belfield et al. (2006) performed a cost-benefit analysis of the Perry Preschool cohort

using data collected from the participants at age 40 and, in addition to finding higher incomes,

asset ownership, and lower crime rates among program participants as Schweinhart and Weikart

(1980, 1993) had, Belfield et al. also found higher presences of positive health indicators among

program participants. Belfield et al. (2006) noted 43% of the program group reported they

stopped working due to health reasons, as opposed to 55% of the control group (178). Twenty

percent reported health problems as opposed to 29% of the program group, fewer program

participants smoked than their control group counterparts (42% vs. 55%), and fewer program

participants required treatment for substance abuse than their control group peers (22% vs. 34%)

(id.).

In addition to the two differing schools of thought regarding the Perry results, many

scholars criticize the study for its research methods and ensuing implications. Many call the Perry

Preschool Study a “hothouse” program (Whitehurst, 2013, 2014; Besharov, 2008; Olsen, 1999);

that is, a small program with a highly controlled environment and experienced teachers

implementing the program. These programs, critics argue, do not generalize to real-world settings

since resources are more scarce (hothouse programs are costly), programs would operate in less

10

controlled settings, teachers would not have as much training, and parents do not have access to

the resources and personal attention they received during a hands-on, highly structured

intervention. Critics level these charges at other well-known early childhood education studies as

well.

The Carolina Abecedarian Project (ABC) is another seminal early childhood education

study. ABC sought to determine the impacts of a “stimulating early childhood environment”

(Campbell et al., 2014, 1478) on disadvantaged young children. This program placed children in

an all-day child care program, during which children had exposure to “cognitive and social

stimulation” as well as preventative health screenings and meals provided at the center. Adults

participated heavily in the program through supervision and instruction. Part of the program also

consisted of sending learning materials home with children (when they reached ages 6-8) so

parents could continue children’s learning processes. The initial evaluation randomized children

into a program group (who attended the ABC Program) and a control group. Campbell et al.

(2014) collected biomedical data from the ABC Study subjects when they reached their mid-30s,

and found better health indicators among the ABC participants, thereby giving them lower

likelihoods of non-communicable, preventable diseases. Campbell et al. (2014) found lower

blood pressures among ABC Program participants, which were significant at the five percent

level. ABC participants also had lower HDL cholesterol rates and lower obesity rates as well.

Again, critics point out Abecedarian was a “hothouse” program and cannot generalize to the

broader population, especially since the demographics of children in America have changed since

the ABC Study started. For example, Whitehurst (2013) points out neither the Perry Preschool

Study nor the ABC Study included Hispanic children, yet Hispanic children now comprise nearly

half of four-year-olds in Head Start. Therefore, generalizing the results of small-scale studies

from decades ago to present-day children requires “a prodigious leap of faith” (Whitehurst,

2013).

11

Ou and Reynolds (2006) studied another well-known early childhood education program,

the Chicago Child-Parent Center (CPC) Preschool Program. This comprehensive early childhood

education program operated for 3 and 4 years olds in high-poverty neighborhoods in Chicago

where Head Start was absent. Parental involvement and health services comprised aspects of the

CPC Program with the intentions of promoting school readiness and longer-term academic

achievement. Ou and Reynolds (2006) found significantly higher levels of education attainment

among the CPC Program group than the comparison group. However, these levels were still

below national averages of education attainment. Other scholars, though, have criticized CPC for

its research design. The CPC researchers used propensity score matching to create a comparison

group, which has weaker internal validity than a control group created by randomization.

According to Whitehurst (2014) “[s]trong external validity is undermined by weak internal

validity, so the ability of CPC to generalize to the broader population as an effective early child

care program is weaker than its RCT counterparts (i.e., the Perry Preschool Project and ABC

Study).

Shonkoff and Phillips (2000) cite research indicating the effects of aspects of parent

engagement on young children. For example, a strong association exists between the amount of

time a mother spends talking to her child and that child’s vocabulary and early learning test

scores(139). These children who had high vocabularies at age 3 also performed well on

vocabulary and reading tests in 3rd grade (id.), thereby indicating the relationship between

parental behaviors during early childhood and a child’s elementary school performance. Shonkoff

and Phillips (2000) also cite research indicating parents who form “secure attachment

relationships” (169) with their children in infancy helps children form healthy friendships with

other children in the toddler years; these children would get along with peers and have increased

“social competence” (Pastor, 1981, as cited in id.). Conversely, infants who do not have these

relationships with their parents have limited social competence and later become more hostile

(id., 170).

12

Despite criticisms of these seminal studies, they still influence early childhood education

programming today. These critics do not deride early childhood education; rather they seek to

determine causal chains and “be more picky” with evidence (Whitehurst, 2014). Critics caution

against broad scale-up and generalization of programs with limited results, but RTT-ELC is not a

one-size-fits-all early childhood education program. Rather, RTT-ELC allows states to choose

which aspects of these well-known programs (such as health promotion and family engagement)

it wants to address in order to best suit the needs of children in their respective states. Critics do

agree “remedial assistance” and increased parental involvement can help low-income children

based on expert evidence (Besharov, 2005, 2; 2008). However, many low-income parents do not

know how to become more involved in their children’s education and development (Besharov,

2008) and through health promotion and family engagement sub-criteria of RTT-ELC, ACF seeks

to remedy this problem as part of helping children prepare for kindergarten.

Research Questions and Rationale

This implementation evaluation will serve two purposes. First, it will provide an across-the-board

snapshot of grantee states to inform ACF of what grantee states have done with RTT-ELC funds. With

this snapshot-in-time, ACF staff will more easily be able to compare and contrast states’ health and

family engagement initiatives, see RTT-ELC trends, and see what challenges grantee states face. This

more in-depth, comparative analysis of grantee states’ initiatives will help inform ACF of what is

happening across the grantee states and provide information about other states’ practices. Knowledge of

other states’ practices can help ACF which has grantees in other states with other early childhood

programs to provide new insights or technical assistance to their respective states; these states might not

otherwise know about the practices identified in this evaluation.

Secondly, this implementation evaluation will determine to what extent states’ program designs

match their performance reports with regards to child health and family engagement initiatives. With this

type of evaluation, ACF can better understand where gaps exist in RTT-ELC. Additionally, the federal

13

government has invested over $1 billion of taxpayer money into RTT-ELC; an implementation evaluation

will determine how effectively states are using these funds and ensure accountability.

Methodology

This evaluation will look at the states that have addressed C(3) and/or C(4), either as separate

sections or together. Table 1 shows the states in each phase the addressed health promotion and family

engagement.

Table 1: RTT-ELC States that Address Health Promotion and Family Engagement

Phase I (2012 – 2015)

State Health Family Engagement

California X

Delaware X

Maryland X X

Massachusetts X

North Carolina X X

Washington X

Phase II (2013 – 2016) Oregon X

Wisconsin X

Phase III (2014 – 2017) Georgia X

Kentucky X

Michigan X X

New Jersey X X

Pennsylvania X

Vermont X

14

First, I examine these states’ RTT-ELC grant applications, which list states’ priorities and tell ACF what

they plan to do in the areas of health and family engagement with grant funds. If a state’s application says

it will use funds for an initiative, scale-up an existing initiative, or achieve certain program milestones

within a given timeframe, I take these to mean they plan on implementing the specified initiatives and

meeting certain milestones. Many states detail existing initiatives to provide ACF with a picture of how

early learning programs operate in their states, and these are excluded from my analysis since these

initiatives have already been completed. Initiatives which states plan for the future that they say do not

require RTT-ELC funds are also excluded from my analysis.

I then compare initiatives stated in applications to states’ annual performance reports (APRs) to

determine if states achieved what they set out to do with their grants. I obtain additional information from

states’ RTT-ELC websites as needed. These sites inform relevant stakeholders and the general public

about RTT-ELC and will assist with background information on states’ early child education programs.

Additional sources may come from the Office of Head Start (which also operates under ACF) since ACF

uses resources from Head Start to inform their family engagement definitions and practices in other

offices and programs.

States’ specified that health and family engagement goals in their applications will serve as the

evaluative criteria in this implementation evaluation. In other words, I evaluate states’ performance based

on what they stated they would do with RTT-ELC funds in their respective applications. I compare the

performance reports to these specified goals to determine if states succeeded in implementing the

initiatives in line with those goals.

Rankings System for States

To indicate how well states met their targets, I developed a rankings system. This rankings

system though, only provides a snapshot in time of the health promotion and family engagement

landscapes in RTT-ELC states; it is not a definitive analysis. Rather, it is employed for more easily

understanding what states are doing with RTT-ELC funds to promote health and family engagement.

15

States are given a high ranking if their APR indicates they have successfully completed a

specified initiative and they have met or exceeded the performance measures they set in their applications.

In tables, high is indicated with the color red. States are given a moderate ranking if, based on their APR,

they have completed some of the specified initiatives and met some of the performance measures or fell

slightly short. In tables, yellow indicates a moderate ranking. States are given a low ranking if their APR

demonstrates they have not completed the initiatives specified in their applications. States that have not

met their performance measures, or have missing data on the performance measures, are also given a low

ranking. Additionally, if evidence of an initiative is absent from an APR, I designate a low ranking as

well. “N/A,” or not applicable, is designated to states that chose not to address a certain sub-criterion.

Some states fall on the cusps between low and moderate and moderate and high. If a state tends

towards a higher or lower end within a ranking, a plus (+) or minus (-) sign follows their ranking. For

example, a state could be ranked moderate-plus if it had some successes in implementing its initiatives

and seems to be making good progress towards fulfilling these initiatives. A state could receive a high-

minus rating if it has succeeded in most of its initiatives, but fallen slightly short in some.

In the tables in the body of the paper, I have grouped rankings by sub-criteria. That is, each state

will receive a high, moderate, or low ranking based on the averages of the successes of their initiatives.

Phase One States

Among the states in Phase One that addressed health and family engagement, several themes

emerge. All states appeared to have strong existing infrastructure in child health and early learning as well

as state leaders committed to improving early childhood education. In California, the receipt of RTT-ELC

aligned with the election of a new Governor, Jerry Brown, as well as a new Superintendent of Public

Instruction (SPI), and various State Board of Education members, thereby ushering in a climate ripe for

reform. The SPI and Governor Jerry Brown demonstrated commitments to education, even in spite of

California’s projected budget shortfall. In Delaware and North Carolina, states had started undertaking

reforms before RTT-ELC and sought to grant funds to scale up existing successful projects. Delaware had

16

started rethinking behavioral health in young children long before RTT-ELC, and North Carolina had

existing programs for which it did not require RTT-ELC funds. Around the time of Phase I of RTT-ELC,

states were experiencing budget cuts

The following section displays states’ rankings in each sub-criteria in table format and proceeds

to describe the justifications for each state’s rankings in the sub-criteria. For further details on states’

specific initiatives and to what extent they fulfilled them, see Appendix A.

Table 2

Phase I States that Addressed Health Promotion

Table 3 Phase I States that Addressed Family Engagement

17

California

California received a moderate ranking in the first health promotion sub-criterion. California

sought to “develop state standards on screening, services, and supports for a comprehensive and

integrated system” in order to promote “best practice standards in screening (State of California, 2011,

147). California’s APR demonstrates a moderate adherence to this application goal in that the state did

develop an early care and education provider screening toolkit for caregivers, which was distributed in

January 2015. California then solicited feedback from caregivers via surveys to determine caregivers’

additional needs in conducting screenings. To accomplish this goal the application had indicated

California would partner with Project LAUNCH,2 but the APR does not mention this partnership.

California also included elements of well-known classroom evaluation instruments (such as the

Environmental Ratings Scales (ERS) and Classroom Assessment Scoring System (CLASS)) in their

TQRIS.

California receives low rankings for both training providers and incorporating nutrition and health

education in its programs. The application spoke of conducting trainings in the I am Moving, I am

Learning program and training nurses in the “Three R’s of Early Childhood Program: Relationships,

Resilience, and Readiness” program, yet the APR showed no evidence of either of these initiatives.

California also indicated it would incorporate health and nutrition programming through the uses of the

ERS and Curriculum Frameworks. Again, California’s APR did not demonstrate how the state achieved

these aims. APRs provide a checklist for states to check off sub-criteria in which they made progress, and

California checked C(3)(c), but beyond checking this box, its APR did not indicate any initiatives in

promoting healthy habits.

Some of California’s initiatives to develop capacity to increase screenings appeared successful.

Regional Learning Consortia members (who received the RTT-ELC grant) “worked to build their cadre of

2 Project LAUNCH (Linking Actions for Unmet Needs in Children’s Health) is an HHS grant administered by the Substance Abuse and Mental Health Services Administration to “promote the wellness of young children from birth to 8 years by addressing the physical, social, emotional, cognitive, and behavioral aspects of their development” (SAMHSA, 2015).

18

trained and reliable assessors in the ERS and CLASS tools” and the TQRIS required level 5 programs to

offer community-based resources to families, including information Strengthening Families, a research-

based approach to building protective factors in families and protecting children from neglect and abuse

(id.). Other mechanisms to build screening capacity, though, did not materialize. The APR does not

mention the uses of the Help me Grow or Power of Preschool (PoP) programs to conduct trainings on

screenings, as the application stated the state would do. In its performance measures, California met its

targets, but says the numbers for the first measure, number of Children with High Needs Screened, were

underreported. Therefore, it is difficult to determine by how many children California exceeded its target

(if it did). California exceeded the numbers for its third measure (number of children enrolled in a

schedule of care) as well as in two of the years in the percentage of children screened who are up to date

in their care schedules. In the second measure, number of children who receive follow-ups or referrals

based on screenings, California did not meet their targets. For these mixed results, they receive a

moderate ranking.

Delaware

Delaware receives a high-minus ranking in creating a progression of health standards. In 2014,

the state added new requirements to Delaware Stars, the state TQRIS. Three and four star programs had to

make use of child development screens, and five start programs had to demonstrate the use of screening

results to inform teaching practice and provide “integrated, individualized teaching” (U.S. Department of

Health and Human Services and U.S. Department of Education, 2014, 6). Other initiatives specified under

C(3)(a) in the application received moderate or low rankings since they appeared partially completed

based on the APR, or were not mentioned at all (see Appendix A).

Delaware mentions several programs it seeks to leverage and scale up in order to fulfill RTT-ELC

initiatives, but how these existing programs are used to implement the initiatives listed in the application

is somewhat unclear, hence the moderate ranking for the second sub-criterion. The Early Child Mental

Health Consultation Program (ECMHC) was made available to all TQRIS programs and more consultants

participated in the Child Adult Relationship Enhancement (CARE) training. Delaware set the goal of

19

training 32 clinicians in Parent Child Interactive Therapy (an evidence-based program), but the APR

mentions only 27 completed this training. Other partnerships Delaware leveraged include a partnership

with Nemours Children’s Health System to train providers in health and nutrition. Under RTT-ELC,

Nemours, along with the Department of Education, the Delaware Institute for Excellence in Early

Childhood (DIEEC), and the Office of Early Learning, developed online, self-paced training modules.

Other tasks for partner institutions that were designated in the application, though, were not in the APR.

The state planned for DIEEC to hire a health consultant to coordinate health promotion activities and

designated DIEEC to track the number of educators participating in trainings, but the APR does not

mention these activities.

Screenings did increase in Delaware, though it is unclear in the APR what partnerships and

structures lead to these screenings. Delaware exceeded the first performance measure in all three years of

the grant thus far, and met or exceeded the third measure as well.

North Carolina

North Carolina focused its health promotion efforts on the last sub-criterion, and the match

between their APR and application was moderate. North Carolina proposed a two-tiered system to

identify and address the health of Children with High Needs. The first tier uses strategies to increase the

number of children screened and referred to services, and the second tier targets what North Carolina calls

the “Transformation Zone,” or “a rural and economically disadvantaged geographic region with a very

high percentage of Children with High Needs” in the northeast corner of the state (State of North

Carolina, 2011, 187). North Carolina succeeded in its initiatives both statewide and in the Transformation

Zone. Statewide, providers increased their uses of proven developmental screening instruments, such as

the Ages and Stages Questionnaire (ASQ) and the Parents Evaluation of Development Skills (PEDS).

Additionally, medical providers were trained to use the Modified Checklist for Autism in Toddlers.

Another statewide success was the establishment of a Child Care Health Consultant (CCHC) model to

train staff in child care settings in health literacy and connecting children to medical homes, and the grant

also allowed North Carolina to expand its Assuring Better Health and Child Development (ABCD)

20

model, which has screened 99% of children in participating medical practices (id.). With RTT-ELC,

ABCD is expanding to more counties and medical providers.

In the Transformation Zone, CCHC is training additional staff specifically for the Zone. The

program NorthEast Connects provides nurses to conduct home visits to connect families to services in the

community and provide further follow-up as needed. As far as performance measures, North Carolina

screened more Children with High Needs than the initial targets in the first two years, but data were

missing for performance measures two and three. For the fourth measure, North Carolina fell short.

North Carolina also addressed family engagement through its two-prong model of targeting

initiatives both statewide and within the very high-needs Transformation Zone. Statewide, North Carolina

leveraged partnerships with Head Start and Early Head Start programs to train early childhood education

providers in regional settings. In the Transformation Zone, the evidence-based Positive Parenting

Program (Triple P) expanded; this “multi-level” program is “designed to prevent or reduce the severity of

behavioral, emotional, and developmental problems in children” (U.S. Department of Health and Human

Services and U.S. Department of Education, 2014, 49). Literacy programs, such as Reach out and Read

and Motheread also expanded in the Transformation Zone.

Massachusetts

Massachusetts is the only state that explicitly addressed family engagement standards, and the

state did so with moderate success through a contract with the University of Massachusetts – Boston, and

they aligned and connected these standards to several other standards and curricula, including the

Massachusetts Early Learning Guidelines for Infants and Toddlers, Massachusetts Curriculum

Frameworks, the Head Start Child Development and Early Learning Framework (HSCDELF).

However, 3 of the 4 initiatives regarding training that were mentioned in the application did not

get mentioned in the APR, which gives Massachusetts a low ranking in the second sub-criterion. The state

earns a moderate ranking for its initiatives to engage families at various levels. The Department of Early

Education and Care worked with partner institutions to develop a financial literacy training and education

programs, both available online and in person. Massachusetts stands out in this regard; no other states

21

explicitly address financial literacy as does Massachusetts. Massachusetts recognizes financial literacy

and education promote family engagement by creating stable home environments for children. Another

successful family engagement initiative Massachusetts has implemented in the Brain Building in

Progress public awareness campaign to promote everyday opportunities to engage families in children’s

development. Buses ran ads, a social media campaign was started, and a Brain Building in Progress

website provided information on child development and a calendar of “brain building” activities and

events within different regions of the state. The state also partnered with the Boston Children’s Museum

to further promote the campaign through exhibits on STEM literacy and interactive activities intended to

prepare young children for Kindergarten. Further details on these initiatives can be found in Appendix A.

Maryland

Maryland receives a moderate-plus ranking in health promotion; while they had some successes

and exceeded performance measures, initial data problems did not make clear these indicators. The initial

application did not contain targets for the third and fourth performance measures, and the initial targets

for performance measures one and two were later lowered. The state imputed these data problems to

duplicated counts of screenings (i.e., a single child may receive multiple screenings) as well as their

education and health departments’ uses of different data systems. However, after the first year, Maryland

revised its target numbers and was able to meet the first and second performance measures; performance

measures three and four, though remained blank. To carry out these screenings, Maryland succeeded in

their specified initiative to train pediatricians in the use of screening tools as well as in addressing early

child mental health.

Maryland also earns a moderate-plus in its efforts to train providers in family engagement

standards and promote family engagement on a wider level. All of its family engagement initiatives

earned at least a moderate rating for completing some of the initiatives from the application. Initiatives

that earned high ratings include the formation of the Maryland Coalition of Family Engagement (which

created an early childhood family engagement framework), the creation of 24 local early childhood

22

advisory councils that implemented the framework, and the local councils’ work in developing and

promoting evidence-based early literacy programs.

Washington

Washington receives moderate rankings in the two sub-criteria it addressed. While the state did

work to incorporate Strengthening Families as a requirement in the TQRIS, as of the APR submission, it

fell short in its target number of facilities that would receive the Strengthening Families training. In the

last family engagement sub-criteria, Washington outlined plans in its application to create an Early

Learning Fellowship program that would train fellows to assist families with early learning needs and

advocate the importance of early learning and development. The Early Learning Fellowship would have

13 fellows, representing the state’s diverse population, by 2011, however, the APR does not mention the

fellowship. Another program mentioned in the APR, Parent Navigators, attempts to do similar work of

the fellowship program through conducting family outreach and creating parent networks. Therefore, the

development of the Early Learning Fellowship is unclear and earns Washington a low ranking for this

initiative. Washington’s other family engagement promotion initiative, though, demonstrated success in

the APR. This initiative, the “Love. Talk. Play.” Parent Campaign, promoted simple, everyday tasks

parents could do to aid in their children’s development. The campaign especially focused on parents of

children ages birth to three and worked to help parents understand their roles as children’s “first and most

important teachers” (U.S. Department of Health and Human Services and U.S. Department of Education,

2014, 63).

Phase Two States

Phase II was RTT-ELC’s smallest phase with only five states, due to a 50% reduction in funding.

As such, only one state (Oregon) chose to address health promotion and only one state chose to address

family engagement (Wisconsin). A common trend across Phase II showed states reapplying for RTT-ELC

funding after initially applying in Phase I and not receiving funding at that time. Oregon was one of these

23

states that reapplied and, like other states, used its Phase I application to influence its Phase II application.

However, Oregon had to change its new application to contend with the reduction in funding.

Within Phase II, each state’s results are indicated below, with further explanations in Appendix

B.

Table 4 Phase II States that Address Health Promotion

Table 5 Phase II States that Address Family Engagement

Oregon

Oregon chose to respond to the fourth criterion and had a moderate match between its APR and

application (again, states have great latitude in determining how they want to address the sub-criteria in

RTT-ELC). Oregon achieved success in adopting the ASQ as the statewide early learning screening

instrument, and used the results and feedback of its field testing of the TQRIS to further improve trainings

for those that administer screenings. Oregon also developed Early Learning Hubs, which operate in local

settings, connect families to nearby services, and conduct screenings.

As far as meeting performance measures, Oregon encountered both success and problems.

Oregon exceeded three of four of its performance measures in the first year, some by several thousand.

However, numbers in all performance measures dropped by large amounts in the second year of RTT-

24

ELC, even though some of these lower numbers still exceeded performance measures. Oregon reports that

it is monitoring these drops to determine the causes and to ensure RTT-ELC is reaching target

populations.

Wisconsin

Wisconsin had some delays in hiring a family engagement coordinator and consultant like they

had specified in their application. These key staff members were eventually hired, but the delay set back

the rest of their work, especially in efforts to create and align family engagement standards. Therefore,

Wisconsin receives a moderate rating in the first family engagement sub-criterion. Wisconsin also

succeeded in creating Communities of Practice across the state to assist with professional development for

early childhood education providers. These meetings take place once a month, and practitioners that

attend share information and practices, network with other providers, and further develop coordination

and streamlining of services.

To promote family engagement statewide, Wisconsin initiated a media outreach campaign. Three

videos were created that emphasized the importance of child development and highlighted and explained

the YoungStar system (the state’s TQRIS). The state also conducted outreach to other youth

organizations, such as tribal caregivers and youth day camps. To especially target its media campaign,

Wisconsin held focus groups with parents to determine what mattered most to them in selecting an early

learning program for their children, and media messages were targeted to these concerns. Wisconsin hired

a local marketing firm to promote YoungStar as well.

Phase Three States

Phase III consists of the most recent iteration of RTT-ELC. Even before receiving RTT-ELC

funding, many states in this phase had already demonstrated commitments to early childhood education.

For example, Michigan appropriated an additional $65 million for early childhood education even before

receiving RTT-ELC. Pennsylvania, another Phase III state, had a TQRIS system already in place for

several years, and Vermont is consistently ranked as one of the top states for child wellbeing. Since Phase

25

III is the most recent cohort, states were able to look back at Phase I and Phase II states for ideas on RTT-

ELC initiatives, and Phase III incorporates a lot previous grantee states’ work in their own applications.

Given that Phase III is the newest phase, it is also the most difficult to of the three phases to

evaluate, since not a lot of data on these states’ initiatives is available as of yet. Phase III states just

implemented their grants a year ago, and many initiatives are planned to last the entire grant period.

Therefore, it is difficult to ascertain fidelity to applications when not a lot of initiatives have started. This

discrepancy may be reflected in some of the ratings.

Phase III of RTT-ELC also added another sub-criteria to health promotion which states had the

option of addressing. This new sub-criteria, C(3)(e) called for “[d]eveloping a comprehensive approach to

increasing the capacity and improve the overall quality of Early Learning and Development Programs to

support and address the social and emotional development (including infant-early child mental health) of

children from birth to age five” (Department of Education and Department of Health and Human

Services, 2013, 54004). Some states chose to respond to this new criterion, as indicated in the tables

below. Further details on Phase III initiatives can be found in Appendix C.

Table 6 Phase III States that Address Health Promotion

26

Table 7

Phase III States that Address Family Engagement

Michigan

Michigan may seem like an anomaly based on its rankings in Table 6, however, this is not

indicative of its success overall in health promotion initiatives, let alone its entire RTT-ELC program. At

the time of this analysis, Michigan had not completed the initiatives outlined in its application received

these rankings because it did not do a lot of the initiatives that were specified in its application, such as

conducting a gap analysis of program standards, and the state demonstrated success in only one

performance measure; the rest had missing data. However, the state was taking steps towards fulfilling its

initiatives in social-emotional and mental health (the newest sub-criterion). At the time of the APR, the

state was finalizing the interagency agreement between its health and education departments for a

program that would train social-emotional consultants.

In family engagement, Michigan had spelled out several initiatives it sought to undertake, but

there was either no mention of these in the APRs or the initiatives have not yet been implemented, again

due to Phase III’s relative newness. Michigan did receive a moderate-minus rating for taking steps

towards a pilot Family Engagement Consultants program, where consultants would work with providers

to effectively engage families and build protective factors. At the submission of the APR, Michigan’s

Office of Great Start developed a Family Engagement Specialist position in order to guide RTT-ELC

family engagement initiatives.

27

New Jersey

New Jersey added a health component to each level of Grow NJ Kids, the state’s TQRIS, but

other intended initiatives under the first sub-criterion showed mixed results. An orientation for Grow NJ

Kids providers were not mentioned in the APR, nor was the quality improvement specialist’s review of

program sites’ improvement plans; both of these tasks were outlined in the application. In the second

performance measure, New Jersey succeeded in developing a Training Academy, which consists of

experts in the components of Grow NJ Kids and offers a wide variety of trainings, but the APR only

mentions trainings in some of the programs listed in the APR; not all. New Jersey is also making progress

in expanding its network of Central Intake Hubs, which are county-level centers that coordinate access to

health and social services and other community supports. At the time of the APR, requests for proposals

were being developed for Central Intake Hubs in six additional counties; these additional hubs would

create a Central Intake Hub in every county in New Jersey.

New Jersey managed to exceed all of its health performance measures in year one; the following

years do not have data since New Jersey is still in year one of its grant. The state still receives a high

ranking in this sub-criterion, though, due to its success so far. New Jersey also chose to address the new

mental health sub-criterion and received a moderate-plus ranking. The Academy has focused on early

childhood mental health and employed evidence-based models to train providers. New Jersey planned for

the Central Intake Hubs to help families and providers to track social-emotional development in children,

but the APR does not list these accomplishments.

Vermont

In the first sub-criterion, Vermont receives a high ranking for implementing nearly all of its

initiatives. Vermont hired key staff to implement health initiatives and train home visitors, expanded

Head Start and Early Head Start, and is implementing a home visiting program, with training to start in

May 2015. Vermont also developed what it calls a C3 team, which is dedicated to implementing the

initiatives under section C3 of RTT-ELC. This team meets monthly to ensure progress on C3 initiatives

and works to ensure alignment between the myriad initiatives that are part of this section.

28

In the remaining four sub-criteria, Vermont receives moderate rankings due to mixed results. The

APR shows no evidence of any of the initiatives regarding Child Care Health Consultants taking place,

but Vermont did succeed in building a web-based platform, called Vermont Insights, and also created a

“warm-line” call option for parents and providers to call with child health and development questions.

Vermont Insights compiles early childhood education data to inform policy discussions as well as

family’s understandings of the early childhood care system. Regional councils have their own web pages

as well to publish newsletters and coordinate with other regional councils.

Initiatives that received moderate rankings include the creation of the Early Childhood

Interagency Coordinating Team, which contains members from different offices involved in RTT-ELC.

Vermont’s application specified a cross-agency group to coordinate screenings and trainings, but the APR

does not clarify whether the Coordinating Team is the group undertaking these efforts. Another moderate

rated initiative was the training of the first cohort of Early Care and Education centers, which assist

parents in understanding their child’s development. ECE centers provide screening and resources to

parents so parents can track their own child’s developmental milestones. These ECE centers use the

Centers for Disease Control’s Learn the Signs: Act Early resource, which provides information on

determining autism in young children, however, the application called for this resource to be translated

into different refugee languages (since Vermont contains a sizeable refugee community), however, the

APR did not show evidence of these translations.

Vermont only exceeded one of its performance measures; the rest it missed by small amounts.

Vermont also chose to address the new sub-criterion that focuses on early child mental health. In this

area, Vermont implemented Early Multi-Tiered System of Supports (MTSS) trainings to providers in

local settings. MTSS is a framework for providing continuous support to students, with support coming

from multiple levels (such as teachers, parents, and other care providers). Vermont also expanded

regional Childhood Advisory Councils and increased their capacity; a full-time manager was hired to

oversee the regional councils and 11 coordinators were hired as well. Vermont’s application indicated the

29

state would conduct regional parent workshops and train providers in the Teaching Pyramid Observation

Tool evaluation tool, but the APR did not show evidence of these initiatives.

Georgia

Georgia receives a high ranking in developing family engagement standards since it succeeded in

its goal to review its Quality Rated (the state’s TQRIS) Program Standards. Georgia convened a Family

Engagement Task Force of diverse stakeholders to create a state definition of family engagement,

identified promising practices, and make recommendations for revisions to the Quality Rated Program

Standards. For the second criterion, Georgia increased the number of programs enrolled in Quality Rated,

with the goal of increasing trainings for the educators in these programs. Even though more programs

enrolled in Quality Rated, the trainings these programs received was unclear, hence the moderate-plus

ranking instead of high (see Appendix C).

In the third criterion, Georgia receives a moderate ranking in part to mixed results, but also

because the state has only very recently implemented some of its initiatives. Some initiatives have only

seen the beginning steps, but these steps indicate progress towards fulfilling the initiatives. One notable

success in this criterion that received a high rating comes from implementing an engagement public

awareness campaign. “Play to Learn” spots are airing on the Georgia Public Broadcasting Network

through June 2015; these ads each highlight a skill within the Georgia Early Learning and Development

Standards.3 Georgia made progress towards implementing a home visiting program called Great Start

Georgia (GSG); the program is slated to start in mid-2015. Webinars have introduced providers to GSG

and the state has worked with other stakeholders to develop a framework for implementing screenings on

a large scale within GSG. These steps towards implementing GSG earned Georgia moderate ratings in the

third family engagement sub-criterion.

3 Early Learning and Development Standards spell out what children should be able to know and do by the time they reach Kindergarten; RTT-ELC also gives states the option to address Early Learning and Development Standards.

30

Kentucky

Kentucky addressed only the last family engagement criterion and succeeded in several initiatives

pertaining to family engagement promotion. Kentucky adopted the Strengthening Families Protective

Factors framework, and added a sixth protective factor: nurturing and attachment. A Strengthening

Families leadership team has worked to incorporate the protective factors within different early learning

settings and programs, and is developing a framework for providers to use to integrate these factors.

Kentucky also succeeded in incorporating family engagement in its TQRIS. All TQRIS programs must

hold at least one family engagement activity per year, and those at the highest level must have at least

four events per year.

Kentucky has also expanded its Toyota Born Learning Academies (later called United Way Born

Learning Academies); these Academies serve as parenting workshops where parents can learn how to turn

everyday moments into opportunities to engage children and assist in their learning and development. At

the time of the APR’s submission, Kentucky hired an Academies coordinator, awarded grants to 24

schools to implement the Academies, and planned for 26 more Academies sites that would receive their

grants in early 2015. The United Way helped create the curriculum for the Academies and Kentucky

planned to transfer administration of the Academies completely over to the United Way by the end of the

grant period. Even though these Academies are in the early stages, planning seems on target (based on the

APR), which earns Kentucky a high ranking it the promoting family support sub-criterion.

Pennsylvania

Pennsylvania listed two initiatives under the first criterion, one which received a high ranking and

one which received a moderate ranking (see Appendix C), which earns the state a moderate-plus ranking

overall. Pennsylvania sought to convene stakeholders to review existing Program Partnership standards to

determine their applicability to key family engagement initiatives and practices, such as home visiting and

early intervention, and the APR indicates this initiative was successful. The state aligned the Program

Partnership standards to Pennsylvania’s Family Engagement Crosswalk, a framework that combines

elements of other protective factor frameworks (including Strengthening Families and the National Parent

31

Teacher Association framework); this initiative receives a high rating. The second initiative, ensuring the

state TQRIS system addresses the seven common elements of family engagement (as identified by the

Crosswalk) has encountered delays, hence earning it a moderate rating. The University of Pennsylvania is

in the process of determining to what extent current standards address the common elements of family

engagement through data collection and analysis on the TQRIS system. The university was supposed to

issue findings and recommendations for improvement in early 2015, but these reports are still pending.

The university’s research informs the rest of the standards revisions process, so the goal of incorporating

the family engagement elements in the TQRIS has not yet reached fruition.

Pennsylvania’s initiatives in the second and third criteria were also successful; the moderate

rating for the second criteria comes from initiatives still being implemented and not yet completed.

However, the state is making appropriate progress on these initiatives. Pennsylvania successfully

distributed 12 grants to local communities for Early Childhood Education Community Innovation Zones

(CIZ). The CIZ would promote family engagement and community partnerships and work closely with

schools to reduce the achievement gap among high needs children. Communities had to demonstrate a

partnership with at least one early education provider and one school that would provide services. Some

programs within CIZ have developed innovate family engagement techniques, like sending parents texts

and pictures from preschool to keep parents aware of what children are doing.

Pennsylvania also held its first statewide Family Engagement Summit, which over 200 family

engagement stakeholders attended to learn more about RTT-ELC, share resources and practices, and

network. Another highly successful initiative was the creation of Keystone Families First (KFF), a

website that helps families make decisions about appropriate early learning programs for their child and

provides information about child development. KFF provides a developmental questionnaire so parents

can gauge their child’s development and see if their child is meeting the correct developmental

milestones. KFF provides family engagement activities by region and provides activities based on

developmental milestones, so parents can pick activities according to where they live as well as by their

32

own child’s developmental needs. Pennsylvania conducted focus group to determine responses to KFF

and have made changes accordingly, and a Spanish version of the website is forthcoming.

Limitations

When evaluating a billion dollar program being implemented in 20 different states, limitations

will arise. The main limitation I encountered in this evaluation was time constraints. Due to time

constraints, I did not have time to conduct qualitative interviews with state-level RTT-ELC actors.

Qualitative interviews could have clarified applications and provided more insight to what was happening

with early childhood education in the states that applications and APRs failed to capture. Speaking with

street-level staff as well, such as social workers, home visiting nurses, or early childhood education

teachers could reveal an additional, deeper layer of analysis regarding how those who most directly

implement RTT-ELC are affected by the policy, what their challenges and successes are with

implementation, and how they feel about the program.

With time constraints, I also only looked at one year of APRs. Phase I states have three APRs

thusfar (one submitted for every year since they’ve implemented the grant), Phase II states have two, and

Phase III have only submitted their first APRs. For all states, I looked the most recent APRs, submitted in

late 2014/early 2015, in order to have the most up to date performance report of what states had achieved

thusfar. Looking at all years could have provided insight to more trends in states’ initiatives and further

explanation on initiatives. Additionally, some states that achieved an initiative or milestone in previous

APRs perhaps could have neglected to mention it in the most recent APR; therefore, my limited view of

APRs could have ignored some work that states have accomplished, or states could have been given a low

ranking for not mentioning an initiative when in fact, they had already completed it in a previous grant

year and therefore saw no need to mention it in another APR. Additionally, the APRs I did analyze were

in rough draft form; revisions will be submitted in May 2015. Revisions could contain additional

performance data, program successes, or other clarifications that would add to my analysis, but this

information is not included since I do not yet have access to these revised APRs.

33

Time also limited the number of initiatives within RTT-ELC I could study. Health promotion and

family engagement are two small components of a large federal program. RTT-ELC addresses myriad

aspects of early childhood education, from workforce preparation, to data systems, assessments, and

more. Therefore, judging RTT-ELC as a whole based on my analysis of two components within the

program does not provide an accurate assessment of the program’s success.

The newness of RTT-ELC also presents limitations in this analysis. Phase I states are just

entering the final year of their grants, and Phase III states just implemented their grants a year ago.

Evaluating programs still in progress can be difficult since states are still implementing aspects of the

program, and with Phase III, only a few initiatives have been implemented. Some states have experienced

delays, which further pushes back initiatives and leaves less to evaluate. Within the coming years as more

initiatives develop and grant periods conclude, RTT-ELC will require more analysis and evaluation to

determine more complete, accurate accounts of what states have done with RTT-ELC funds and if the

program has succeeded in preparing low-income children for kindergarten.

Another limitation of this study it’s the potential skewedness of states’ ratings. States have

different sized programs; for example, Washington has four family engagement initiatives, while

Massachusetts has 17 (see Appendix A). So, if a state only has one initiative in a sub-criterion, and this

initiative is ranked high, the state appears more successful than, say, a state with a mix of high, moderate,

and low-ranked initiatives under one sub-criterion. This unevenness must be taken into consideration

when viewing my analysis, and, again, this limitation reveals my analysis is not a definitive evaluation of

RTT-ELC.

Recommendations

Even with the limitations of my analysis, states do appear to be making progress in several

aspects of RTT-ELC. In order to better track state success and possibly improve RTT-ELC (based on my

relatively narrow view of the program), states and ACF could enact some reforms. These reforms include:

34

• A streamlined reporting system. Information appears to be lost between the application and the

APR. If states had a streamlines system through which they could specifically describe an

initiative and its components (rather than wordily describing initiatives in the applications and

maybe mentioning them in the APR), both states and ACF could track the progress of an

initiative. Some states in this analysis received low ratings because completion of an initiative

was not explicitly clear; the APR indicated states had made progress overall in implementing

RTT-ELC, but the mechanisms through which states implemented RTT-ELC programs

• The use of logic models. Logic models would also allow for better tracking of projects and

initiatives. Some states’ applications contain logic models, but not all of them; the applications

with logic models seemed easier to read and understand. Logic models would clarify where states

are in their processes of implementing initiatives, which would help ACF better ascertain states’

progress and would help states stay on track to implementing initiatives and better be able to

report progress.

• Greater coordination and oversight within states. While stakeholders within states appear to

coordinate to implement RTT-ELC, coordination during the reporting process could also work to

better track of all the initiatives taking place. This coordination could track what grant funding is

being used for, through what mechanisms funds are being used (i.e., existing programs and

structures or new ones), and ensure this information is conveyed in the APR to inform ACF staff.

States seem to be making great strides in expanding early learning programs and ensuring these programs

meet the needs of low-income children, but further evaluation will be needed to determine longer-term

outcomes of RTT-ELC. To conduct evaluation, clearer and more concrete data will be needed, and to

obtain this data, more communication and coordination between those implementing RTT-ELC in the

states and those reporting on the grant would help both ACF, ED, and the broader public ascertain RTT-

ELC’s lasting outcomes and successes.

35

References

Belfield, C. R., Nores, M., Barnett, S., and Schweinhart, L. (2006). The High/Scope Perry Preschool

Program: Cost-Benefit Analysis Using Data from the Age-40 Followup. The Journal of Human

Resources, 41(1), 162-190.

Besharov, D. and Ramsey, C. (2008). Preschool Puzzle. The Hoover Institution. Retrieved from

http://object.cato.org/sites/cato.org/files/pubs/pdf/pa333.pdf

Besharov, D. J. (2005). Head Start’s Broken Promise. The American Enterprise Institute for Public Policy

Research. Retrieved from

http://www.welfareacademy.org/pubs/childcare_edu/brokenpromise.pdf

Campbell, F., Coniti, G., Heckman, J. J., Moon, S. H., Pinto, R., Pungello, E., and Pan, Y, (2014). Early

Childhood Investment Substantially Boosts Adult Health. Science, 343(28), 1478-1485.

Olsen, D. A. (1999). Universal Preschool is not the Golden Ticket: Why the Government Should Not

Enter the Preschool Business. Policy Analysis, 333(1999), Retrieved from

http://object.cato.org/sites/cato.org/files/pubs/pdf/pa333.pdf

Schweinhart, L. J. and Weikart, D. P. (1993). Success by Empowerment: The High/Scope Perry

Preschool Study Through Age 27. Young Children, 48(1), 54-58.

Schweinhart, L. J. and Weikart, D. P. (1980). Young Children Grow Up: The Effects of the Perry

Preschool Program on Youths Through Age 15. Ypsilanti: High/Scope Educational Research

Foundation.

Shonkoff, J. P. and Phillips, D. A., eds. (2000). From Neurons to Neighborhoods: The Science of Early

Childhood Development. The Board on Children, Youth, and Families, Committee on

Integrating the Science of Early Childhood Development. Washington, D.C.: National Academies

Press.

The State of California. (2011). Race to the Top Early Learning Challenge Application. Retrieved from

http://www2.ed.gov/programs/racetothetop-earlylearningchallenge/applications/california.pdf

36

The State of Delaware. (2011). Race to the Top – Early Learning Challenge Application for Initial

Funding. Retrieved from http://www2.ed.gov/programs/racetothetop-

earlylearningchallenge/applications/delaware.pdf

The State of Georgia. (2013). Race to the Top – Early Learning Challenge: Application for Initial

Funding. Retrieved from http://www2.ed.gov/programs/racetothetop-

earlylearningchallenge/applications/2013-georgia.pdf

The State of Kentucky. (n. d.) Kentucky ALL STARS: Accelerated Learning Statewide through an

Advanced Ratings System. Retrieved from http://www2.ed.gov/programs/racetothetop-

earlylearningchallenge/applications/2013-kentucky.pdf

The State of Maryland. (2011). Race to the Top – Early Learning Challenge Application for Initial

Funding. Retrieved from http://www2.ed.gov/programs/racetothetop-

earlylearningchallenge/applications/maryland.pdf

The State of Massachusetts. (2011). From Birth to School Readiness: The Massachusetts Early Learning

Plan, 2012 – 2015. Grant Proposal: Race to the Top – Early Learning Challenge. Retrieved from

http://www2.ed.gov/programs/racetothetop-

earlylearningchallenge/applications/massachusetts.pdf

The State of Michigan. (2011). Race to the Top – Early Learning Challenge: Application for Initial

Funding. Retrieved from http://www2.ed.gov/programs/racetothetop-

earlylearningchallenge/applications/2013-michigan.pdf

The State of New Jersey. (2013). (2013). Race to the Top – Early Learning Challenge: Application for

Initial Funding. Retrieved from http://www2.ed.gov/programs/racetothetop-

earlylearningchallenge/applications/2013-new-jersey.pdf

The State of North Carolina. (2011). Race to the Top – Early Learning Challenge Application for Initial

Funding. Retrieved from http://www2.ed.gov/programs/racetothetop-

earlylearningchallenge/applications/north-carolina.pdf

37

The State of Oregon. (2012). Oregon Race to the Top – Early Learning Challenge Phase II Application.

Retrieved from http://www2.ed.gov/programs/racetothetop-

earlylearningchallenge/applications/oregon-2.pdf

The State of Pennsylvania. (2013). Race to the Top – Early Learning Challenge: Application for Initial

Funding. Retrieved from http://www2.ed.gov/programs/racetothetop-

earlylearningchallenge/applications/2013-pennsylvania.pdf

The State of Vermont. (2013). Application for Initial Funding Under Race to the Top – Early Learning

Challenge. Retrieved from http://www2.ed.gov/programs/racetothetop-

earlylearningchallenge/applications/2013-vermont.pdf

The State of Washington. (2011). Race to the Top – Early Learning Challenge Grant Application.

Washington State Department of Early Learning. Retrieved from

http://www2.ed.gov/programs/racetothetop-earlylearningchallenge/applications/washington.pdf

The State of Wisconsin. (2012). Race to the Top – Early Learning Challenge Application for Phase 2

Funding. Retrieved from http://www2.ed.gov/programs/racetothetop-

earlylearningchallenge/applications/wisconsin-2.pdf

U.S. Department of Education and U.S. Department of Health and Human Services. (2013). Applications

for New Awards; Race to the Top – Early Learning Challenge. (2013). Federal Register, 78(169),

53992-54011. Retrieved from http://www.gpo.gov/fdsys/pkg/FR-2013-08-30/pdf/2013-21139.pdf

U.S. Department of Health and Human Services. (2011). The Head Start Parent, Family, and Community

Engagement Framework: Promoting Family Engagement and School Readiness from Prenatal to

Age 8. Retrieved from http://eclkc.ohs.acf.hhs.gov/hslc/standards/im/2011/pfce-framework.pdf.

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: California. Washington, DC: U.S.

Department of Education.

38

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: Delaware. Washington, DC: U.S.

Department of Education.

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: Georgia. Washington, DC: U.S.

Department of Education.

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: Kentucky. Washington, DC: U.S.

Department of Education.

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: Maryland. Washington, DC: U.S.

Department of Education.

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: Massachusetts. Washington, DC:

U.S. Department of Education.

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: Michigan. Washington, DC: U.S.

Department of Education.

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: New Jersey. Washington, DC: U.S.

Department of Education

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: North Carolina. Washington, DC:

U.S. Department of Education.

39

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: Oregon. Washington, DC: U.S.

Department of Education.

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: Pennsylvania. Washington, DC:

U.S. Department of Education.

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: Vermont. Washington, DC: U.S.

Department of Education.

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: Washington. Washington, DC:

U.S. Department of Education.

U.S. Department of Health and Human Services and U.S. Department of Education. (2014). Race to the

Top – Early Learning Challenge Annual Performance Report: Wisconsin. Washington, DC: U.S.

Department of Education.

U.S. Department of Education and U.S. Department of Health and Human Services. (2014). The Race to

the Top – Early Learning Challenge Year Two Progress Report. Washington, DC. Retrieved from

http://www2.ed.gov/programs/racetothetop-earlylearningchallenge/rtt-aprreportfinal112614.pdf

Whitehurst, G. J. (2014). Does Pre-k Work? It Depends How Picky You Are. The Brookings Institution.

Retrieved from http://www.brookings.edu/research/papers/2014/02/26-does-prek-work-

whitehurst

Whitehurst, G. J. (2013). Can We Be Hard-Headed About Preschool? A Look at Universal and Targeted

Pre-K. The Brookings Institution. Retrieved from

http://www.brookings.edu/research/papers/2013/01/23-prek-whitehurst

40