Post on 13-Aug-2020
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 1
NARA Illinois Key Indicator Report for Centers, Group Homes, and Family
Homes
Richard Fiene, Ph.D.
May 30, 2014
ABSTRACT
This report will provide an analysis of Illinois Rules for child care centers, group homes, and family
homes for generating key indicators. There is a brief introduction regarding differential monitoring and
key indicators followed by the generated key indicators.
INTRODUCTION
The key indicator methodology is part of a program monitoring approach called Differential
Program Monitoring which was developed to help streamline the program monitoring of early
care and education programs (please see the appendix for two graphics which help to depict this
relationship). It was first applied in child care licensing but has been used in many other service
types, such as: Head Start Performance Standards, National Accreditation, and child and adult
residential programs. The methodologies are based upon statistical protocols that have been
developed in the tests and measurements literature in which an abbreviated set of items is used to
statistically predict as if the full test was applied. This methodology has been used in regulatory
analysis and is now being proposed for use in Quality Rating and Improvement Systems (QRIS).
TECHNICAL ASPECTS OF THE KEY INDICATOR METHODOLOGY
This section provides the technical and statistical aspects of the key indicator methodology. One
of the first steps is to sort the data into high and low groups, generally the highest and lowest
ratings can be used for this sorting. In very large states this is done on a sampling basis which
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 2
will be described later in the blueprint. Frequency data will be obtained on those programs in the
top level (usually top 20-25%) and the bottom level (usually the bottom 20-25%). The middle
levels are not used for the purposes of these analyses. These two groups (top level & the bottom
level) are then compared to how each program scored on each item within the specific
assessment tool (see Figure 1).
Figure 1 Providers In
Compliance
or Top 25%
Programs Out
Of Compliance
or Bottom 25%
Row Total
Highest level
(top 20-25%)
A B Y
Lowest level
(bottom 20-25%)
C D Z
Column Total W X Grand Total
Once the data are sorted in the above matrix, the following formula (Figure 2) is used to
determine if the standard is a key indicator or not by calculating its respective Phi coefficient.
Please refer back to Figure 1 for the actual placement within the cells. The legend (Figure 3)
below the formula shows how the cells are defined.
Figure 2 – Formula for Phi Coefficient
Figure 3 – Legend for the Cells within the Phi Coefficient
A = High Group + Programs in Compliance on Specific Compliance Measure. B = High Group + Programs out of Compliance on Specific Compliance Measure. C = Low Group + Programs in Compliance on Specific Compliance Measure. D = Low Group + Programs out of Compliance on Specific Compliance Measure. W = Total Number of Programs in Compliance on Specific Compliance Measure. X = Total Number of Programs out of Compliance on Specific Compliance Measure. Y = Total Number of Programs in High Group. Z = Total Number of Programs in Low Group.
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 3
Once the data are run through the formula in Figure 2, the following chart (Figure 4) can be used
to make the final determination of including or not including the item as a key indicator. Based
upon the chart in Figure 4, it is best to have a Phi Coefficient approaching +1.00 however that is
rarely attained with licensing data but has occurred in more normally distributed data.
Continuing with the chart in Figure 5, if the Phi Coefficient is between +.25 and -.25, this
indicates that the indicator is unpredictable in being able to predict overall compliance with the
quality rating assessment tool. Either a false positive in which the indicator appears too often in
the low group as being in compliance, or a false negative in which the indicator appears too often
in the high group as being out of compliance. This can occur with Phi Coefficients above +.25
but it becomes unlikely as we approach +1.00 although there is always the possibility that other
standards/rules/regulations could be found out of compliance (this was demonstrated in a study
conducted by the author. Another solution is to increase the number of key indicators to be
reviewed but this will cut down on the efficiency which is desirable and the purpose of the key
indicators.
The last possible outcome with the Phi Coefficient is if it is between -.26 and -1.00, this indicates
that the indicator is a terrible predictor because it is doing just the opposite of the decision we
want to make. The indicator would predominantly be in compliance with the low group rather
than the high group so it would be statistically predicting overall non-compliance. This is
obviously something we do not want to occur.
Figure 4 – Thresholds for the Phi Coefficient
Phi Coefficient Range Characteristic of Indicator Decision
(+1.00) – (+.26) Good Predictor Include
(+.25) – (-.25) Unpredictable Do not Include
(-.26) – (-1.00) Terrible Predictor Do not Include
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 4
RESULTS
Key indicators for child care homes (Please see the Appendix - Figure 7 for Phi
Coefficients):
Section 406.8 General Requirements for Day Care Homes
a) The physical facilities of the home, both indoors and outdoors, shall meet the following requirements for safety
to children.
1) The home shall have a first aid kit consisting of adhesive bandages, scissors, thermometer, non-permeable
gloves, Poison Control Center telephone number (1-800-222-1222 or 1-800-942-5969), sterile gauze pads, adhesive
tape, tweezers and mild soap.
18) There shall be written plans for fire and tornado emergencies. Caregivers and assistants in the home shall be
familiar with these plans.
A) The fire evacuation plan shall identify the exits from each area used for child care and shall specify the
evacuation route.
B) The fire evacuation plan shall identify a safe assembly area outside of the home. It shall also identify a near-by
indoor location for post-evacuation holding if needed.
C) The fire evacuation plan shall require that the home be evacuated before calling the local emergency number
911.
D) The written tornado plan shall specify what actions will be taken in the event of tornado or other severe weather
warning, including designation of those areas of the home to be used as the safe spots.
23) The licensee shall inspect the home daily, prior to arrival of children, ensuring that escape routes are clear
and that exit doors and exit windows are operable. A log of these daily inspections shall be maintained for at least
one year, and shall be available for review. The log shall reflect, at minimum, the date and time of each inspection
and the full name of the person who conducted it.
24) The licensee shall hold monthly fire inspections of the day care home.
Section 406.9 Characteristics and Qualifications of the Day Care Family
a) No individual may receive a license from the Department when the applicant, a member of the household age 13
and over, or any individual who has access to the children cared for in a day care home, or any employee of the day
care home, has not authorized the background check required by 89 Ill. Adm. Code 385 (Background Checks) and
been cleared in accordance with the requirements of Part 385.
t) The caregivers shall complete 15 clock hours of in-service training per licensing year in accordance with the
requirements in Appendix D of the rules.
1) Such training may be derived from programs offered by any of the entities identified in Appendix D of the rules.
2) Courses or workshops to meet this requirement include, but are not limited to, those listed in Appendix D of the
rules.
3) The records of the day care home shall document the training in which the caregiver has participated, and these
records shall be available for review by the Department.
4) Caregivers obtaining clock hours in excess of the required 15 clock hours per year may apply up to 5 clock hours
to the next year's training requirements.
Section 406.12 Admission and Discharge Procedures
b) Prior to acceptance of a child for care,
3) The caregiver shall require that the parent or guardian provide a certified copy of the child’s birth certificate. The
caregiver:
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 5
A) Shall provide a written notice to the parent or guardian of a child to be enrolled for the first time that within 30
days after enrollment the parent or guardian shall provide a certified copy of the child’s birth certificate or other
reliable proof of identity and age of the child.
i) The caregiver shall promptly make a copy of the certified copy and return the original certified copy to the parent
or guardian.
ii) If a certified copy of the birth certificate is not available, the parent or guardian must submit a passport, visa or
other governmental documentation as proof of the child’s identity and age and an affidavit or notarized letter
explaining the inability to produce a certified copy of the birth certificate [325 ILCS 50/5] .
iii) The notice to parent or guardian shall also indicate that the caregiver is required by law to notify the Illinois
State Police or local law enforcement agency if the parent or guardian fails to submit proof of the child’s identity
within the 30 day time frame;
h) All day care homes shall have a written policy that explains the actions the provider will take if a parent or
guardian does not retrieve, or arrange to have someone retrieve, his or her child at the designated, agreed upon time.
The policy shall consist of the provider’s expectations, clearly presented to the parent or guardian, in the form of a
written agreement that shall be signed by the parent or guardian, and shall include at least the following elements:
The consequences of not picking up the child on time, including:
Amount of late fee, if any, and when those fees begin to accrue;
The degree of diligence the provider will use to reach emergency contacts, e.g., number of attempted phone calls to
parents and emergency contacts, requests for police assistance in finding emergency contacts; and
Length of time the facility will keep the child beyond the pick-up time before contacting outside authorities, such as
the child abuse hotline or police.
Emphasis on the importance of having up-to-date emergency contact numbers on file.
Acknowledgement of the provider’s responsibility for the child’s protection and well-being until the parent or
outside authorities arrive.
A reminder to the day care provider that the child is not responsible for the situation. All discussions regarding these
situations shall be with the parent or guardian, never the child.
Section 406.14 Health, Medical Care and Safety
c) A medical report, on forms prescribed by the Department, shall be on file for each child, on the first day of care,
and shall be dated no earlier than 6 months prior to enrollment.
1) The medical report shall be valid for 2 years, except that subsequent examinations for school-age children shall
be in accordance with the requirements of Section 27.8-1 of the School Code [105 ILCS 5/27-8.1], provided copies
of the exam are on file at the facility.
2) If the child is in a high risk group, as determined by the examining physician, a tuberculin skin test by the
Mantoux method and the results of that test shall be included in the initial examination for all children who have
attained one year of age, or at the age of one year for children who are enrolled before their first birthday. The
tuberculin skin test by the Mantoux method shall be repeated when the children in high-risk groups begin
elementary and secondary school.
3) The initial examination shall show that children from 6 months through 6 years of age have been screened for
lead poisoning for children residing in an area defined as high risk by the Illinois Department of Public Health in its
Lead Poisoning Prevention Code (77 Ill. Adm. Code 845) or that a lead risk assessment has been completed for
children residing in an area defined as low risk by the Illinois Department of Public Health.
4) The report shall indicate that the child has been immunized as required by the rules of the Illinois Department of
Public Health for immunizations (77 Ill. Adm. Code 695). These required immunizations are poliomyelitis,
measles, rubella, diphtheria, mumps, pertussis, tetanus, hepatitis B, haemophilus influenza B, and varicella
(chickenpox) or provide proof of immunity according to requirements in Part 695.50 of the Department of Public
Health.
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 6
Key indicators for Group Child Care Homes (Please see the Appendix - Figure 7 for Phi
Coefficients):
Section 408.35 General Requirements for Group Day Care Home Family
f) The caregivers and all members of the household shall provide medical evidence that they are free of
communicable disease that may be transmitted while providing child care; and, in the case of caregivers, that they
are free of physical or mental conditions that could interfere with child care responsibilities. The medical report for
the caregivers shall be valid for 3 years.
Section 408.45 Caregivers
f) The caregivers shall complete 15 clock hours of in-service training per licensing year in accordance with the
requirements in Appendix G of the rules.
1) Such training may be derived from programs offered by any of the entities identified in Appendix G of the rules.
2) Courses or workshops to meet this requirement include, but are not limited to, those listed in Appendix G of the
rules.
Section 408.60 Admission and Discharge Procedures
j) All group day care homes shall have a written policy that explains the actions the provider will take if a parent or
guardian does not retrieve, or arrange to have someone retrieve, his or her child at the designated, agreed upon time.
The policy shall consist of the provider's expectations, clearly presented to the parent or guardian in the form of a
written agreement that shall be signed by the parent or guardian, and shall include at least the following elements:
1) The consequences of not picking up the children on time, including:
A) Amount of late fee, if any, and when those fees begin to accrue;
B) The degree of diligence the provider will use to reach emergency contacts, e.g., number of attempted phone calls
to parents and emergency contacts, requests for police assistance in finding emergency contacts; and
C) Length of time the facility will keep the child beyond the pick-up time before contacting outside authorities,
such as the child abuse hotline or police.
2) Emphasis on the importance of having up-to-date emergency contact numbers on file.
3) Acknowledgement of the provider's responsibility for the child's protection and well-being until the parent or
outside authorities arrive.
4) A reminder to staff that the child is not responsible for the situation. All discussions regarding these situations
shall be with the parent or guardian, never with the child.
Section 408.70 Health, Medical Care and Safety
a) A medical report, on forms prescribed by the Department, shall be on file for each child, on the first day of care,
and shall be dated no earlier than 6 months prior to enrollment.
1) The medical report shall be valid for 2 years, except that subsequent examinations for school-age children shall
be in accordance with the requirements of Section 27-8.1 of the School Code [105 ILCS 5/27-8.1], provided copies
of the exam are on file at the facility.
2) If the child is in a high risk group, as determined by the examining physician , a tuberculin skin test by the
Mantoux method and the results of that test shall be included in the initial examination for all children who have
attained one year of age, or at the age of one year for children who are enrolled before their first birthday. The
tuberculin skin test by the Mantoux method shall be repeated when children in high risk groups begin elementary
and secondary school.
3) The initial examination shall show that children from 6 months through 6 years of age have been screened for
lead poisoning for children residing in an area defined as high risk by the Illinois Department of Public Health in its
Lead Poisoning Prevention Code (77 Ill. Adm. Code 845) or that a lead risk assessment has been completed for
children residing in an area defined as low risk by the Illinois Department of Public Health.
4) The report shall indicate that the child has been immunized as required by the rules of the Illinois Department of
Public Health for immunizations (77 Ill. Adm. Code 695). These required immunizations are poliomyelitis,
measles, rubella, diphtheria, mumps, pertussis, tetanus, hepatitis B, haemophilus influenza B, and varicella
(chickenpox) or provide proof of immunity according to requirements in Part 695.50 of the Department of Public
Health.
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 7
Section 408.120 Records and Reports a) A facility shall maintain a record file on the children enrolled.
1) A written application for admission of each child shall be on file with the signature of the parent or guardian.
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 8
Key indicators for Child Care Centers (Please see the Appendix-Figure 7 for Phi Coefficients):
Section 407.100 General Requirements for Personnel
f) Staff shall have physical re-examinations every two years and whenever communicable disease or illness is
suspected.
Section 407.120 Personnel Records
a) A confidential file shall be maintained on each staff person and contain at least the following information:
1) A copy of a form prescribed by the Department which contains information on persons employed in the day care
center;
3) Three written character references, verified by the day care center;
4) Proof of educational achievement as required for the individual's position. Foreign credentials require additional
documentation providing a statement of the equivalency in the U.S. educational system;
Section 407.250 Enrollment and Discharge Procedures
d) The facility shall distribute a summary of the licensing standards, provided by the Department, to the parents or
guardian of each child at the time that the child is accepted for care in the facility. In addition, consumer information
materials provided by the Department including, but not limited to, information on reporting and prevention of child
abuse and neglect and preventing and reporting communicable disease shall be distributed to the parents or guardian
or each child cared for when designated for such distribution by the Department.
Section 407.260 Daily Arrival and Departure of Children
f) All day care centers shall have a written policy that explains to parents and staff the actions the center will take if
a parent or guardian does not pick up, or arrange to have someone pick up, his or her child at the designated, agreed
upon time. The policy shall consist of the provider's expectations clearly presented to the parent or guardian in the
form of a written agreement that shall be signed by the parent or guardian and shall include at least the following
elements:
1) The consequences of not picking up children on time shall be precisely communicated to parents, for example:
A) Amount of late fee, if any, and when those fees begin to accrue.
B) The degree of diligence the provider will use to reach emergency contacts, e.g., number of attempted phone calls
to parents and emergency contacts, requests for police assistance in finding emergency contacts, and so forth.
C) Length of time the facility will keep the child beyond the pick-up time before contacting outside authorities,
such as, the child abuse hotline, police, and so forth.
2) Emphasis on the importance of having up-to-date emergency contact numbers on file.
3) Acknowledgement of the provider’s responsibility for the child’s protection and well-being until the parent or
outside authorities arrive.
4) A policy that staff shall not hold the child responsible for the situation and that discussion of this issue will only
be with the parent or guardian and never with the child.
Section 407.270 Guidance and Discipline
a) The day care center shall develop a guidance and discipline policy for staff use that is also provided to parents.
Staff shall sign the guidance and discipline policy at the time of employment and parents shall sign the policy when
their child is enrolled. The policy shall include:
1) A statement of the center's philosophy regarding guidance and discipline;
2) Information on how discipline will be implemented by staff;
3) Information on how parents will be involved in the guidance and discipline process;
4) Information on how children will be involved in the guidance and discipline process; and
5) Written procedures for termination of a child's enrollment in the day care center because of disciplinary issues.
Section 407.310 Health Requirements for Children
a) A medical report on forms prescribed by the Department shall be on file for each child.
1) The initial medical report shall be dated less than 6 months prior to enrollment of infants, toddlers and preschool
children. For school-age children, a copy of the most recent regularly scheduled school physical may be submitted
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 9
(even if more than 6 months old) or the day care center may require a more recent medical report by its own
enrollment policy. If a health problem is suspected, the day care center may require additional documentation of the
child's health status.
Section 407.380 Equipment and Materials
b) Such equipment and materials for infants, toddlers and pre-school children shall be provided in the quantity and
variety specified in Appendix A: Equipment for Infants and Toddlers, Appendix B: Equipment for Preschool
Children and Appendix C: Equipment for School-Age Children of the Rules.
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 10
For additional information regarding this Report, please contact:
Richard Fiene, Ph.D., Director/President
Research Institute for Key Indicators (RIKI)
41 Grandview Drive
Middletown, PA. 17057
DrFiene@gmail.com
717-944-5868 Phone and Fax
http://RIKInstitute.wikispaces.com
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 11
Appendix – Figure 5
DIFFERENTIAL MONITORING LOGIC MODEL & ALGORITHM (DMLMA©) (Fiene, 2012): A 4th
Generation ECPQIM – Early Childhood Program Quality Indicator Model
CI x PQ => RA + KI => DM + PD => CO
Definitions of Key Elements:
PC = Program Compliance/Licensing (Health and Safety) (Caring for Our Children) PQ = QRIS/Accreditation/Caregiver/Child Interactions/Classroom Environment Quality (ERS/CLASS/PAS/BAS) RA = Risk Assessment, (High Risk Rules) (Stepping Stones) KI = Key Indicators (Predictor Rules) (13 Key Indicators of Quality Child Care) DM = Differential Monitoring (How often to visit and what to review) PD = Professional Development/Technical Assistance/Training (Not pictured but part of Model) CO = Child Outcomes (Not pictured but part of Model)
Comprehensive
Licensing Tool (CI)
Structural Quality
Program Quality
Tool (PQ) - QRIS
Process Quality
Risk Assessment
Tool (RA)
Key Indicator
Tool (KI)
Differential
Monitoring (DM)
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 12
Appendix – Figure 6 - Licensing Rules, Compliance Reviews, Differential Monitoring, Abbreviated Tools,
Risk Assessment, and Key Indicators
All Licensing Rules – Full
Compliance Reviews
Differential Monitoring
How Often to Visit? What is Reviewed?
Frequency Abbreviated
Tool
More
Often
Less
Often
Risk
Assessment
Weights
Key
Indicators
Predictors
NARA Illinois Key Indicator Report - Fiene RIKI
R e s e a r c h I n s t i t u t e f o r K e y I n d i c a t o r s
Page 13
Appendix -- Figure 7 - Phi Coefficients for the Specific Key Indicators
Family Child Care Homes:
Rule Numbers Phi Content
406.8a1 .34 First Aid Kit
406.8a18 .38 Emergency Plan
406.8a23 .36 Fire Inspection
406.8a24 .35 Log of Home Inspections
406.9a .34 Background Checks
406.9t .38 Caregiver Training
406.12b3 .34 Birth Certificate
40612h .36 Agreement regarding Pick Up
406.14c2 .41 TB Test
406.14c3 .53 Lead Poisoning Screening
406.14c4 .34 Immunizations
Group Child Care Homes:
Rule Numbers Phi Content
408.35f .28 Communicable Diseases
408.45f .31 Caregiver Training
408.60j .33 Agreement Pick Up Policy
408.70a1 .29 Medical Records
408.70a2 .55 TB Test
408.70a3 .51 Lead Poisoning Screening
408.70a4 .35 Immunizations
408.120a1 .37 Written Application Admission for Each Child
Child Care Centers:
Rule Numbers Phi Content 407.100f .35 Staff Physical
407.120a1 .32 CFS-508 Form
407.120a3 .41 Three Written Character References
407.120a4 .34 Proof of Educational Achievement
407.250d .34 Written Standards Given to Parents
407.260f .32 Pick Up Policy
407.270a .32 Discipline Policy
407.310a .44 Medical Report for Each Child
407.380b .34 Equipment Meets Standard Requirements