Web viewA tenth domain, English language acquisition, is included to assess language skills for ......

MEMORANDUM #1

Date: REVISED February 22, 2013

FROM: Gary Resnick, Ph.D., and Pamela Kelley, Ph.D.

TO: Sherri Killins, Commissioner and Jennifer Louis, Project Manager, Massachusetts Early Education and Care

RE: Massachusetts Common Metric Project: Memorandum #1 (REVISED), Identify Common Items within Domains----------------------------------------------------------------------------------------------------------------

Executive Summary

1. An investigation into the extent of conceptual alignment across the three instruments (WSS, GOLD, and COR) suggests a moderate-to-high degree of alignment. In other words, the test developer’s domains were found to be relatively comparable across the three instruments, and there appeared to be good coverage of similar items across all three instruments. For example, of the total 407 items, 75% matched on all three instruments, while only 25% were partial matches (i.e. matched on two or less of the three instruments). This finding supports the plan to move forward with more in-depth analyses.

2. Each instrument was tested separately for internal consistency and reliability by domain. The results suggest a high degree of reliability on all three instruments, with WSS and GOLD having very high reliability (domain alpha coefficients of .93 or greater), while COR was somewhat lower with a wider range (domain alpha coefficients of 0.79 to 0.95). These results suggest that the items used to measure the developer domains were consistent and all tap the same underlying construct.

3. Each instrument was tested separately to assess the extent to which scores were normally distributed. Normally distributed scores can be one indicator of how well an instrument measures or distinguishes between different levels of children’s’ ability. The results were mixed. For example, for WSS, large groups of children had high scores, creating a “ceiling effect” which may indicate the test is not effective at identifying children at different levels of ability for a given developmental domain. For GOLD, normal distributions appeared more consistent, however, a pattern was observed in which scores were clustered in the center of the distribution, suggesting that the test may not distinguish children with particularly low or high ability levels. The COR showed a mixture of both of these distribution patterns. These results suggest that the full range of children’s abilities may not be fully represented by these test domains.

Massachusetts EEC Common Metric Project Memorandum #1 2

4. The assessments were also tested for their ability to distinguish between children’s ability levels by age group. Age group differences in which younger children are rated consistently lower on a test than older children indicate that the test detects maturational differences. Also tested was the interaction effects between age and quartile groups to determine whether younger or older children hold their positions across skill levels. Our testing found mixed results. On the one hand there were significant age differences in the expected developmental progression for many subdomains on all three tests. On the other hand, the age group differences do not consistently hold up across different ability level groups. For example, on the WSS Social Studies subdomain, three-year olds and five-year olds in the lowest ability quartile for their age group had similar scores. These types of mixed findings were evident for at least two subdomains across all three assessments.

5. There are two recommendations that we can make based on the level of missing data found in the dataset, which far exceeds the convention for the amount of missing data “allowed” to continue the sophisticated analyses planned in the next phases of this project. First, the factor analytic analyses should be considered preliminary due to the amount of missing data, and there is concern that there may be enough cases to properly explore some subgroups of the population. A second suggestion is that it would be useful to discuss these missing data issues and potential strategies for improving the quality of future assessment data.

The following memorandum describes the first set of tasks completed under the Common Metric Project. The project comprises a descriptive study of three criterion-referenced child assessment tools in use by early education and care providers within the Commonwealth of Massachusetts. The Massachusetts Department of Early Education and Care made these tools available, along with the training, to allow providers to assess their children’s strengths and challenges, to assist with educational programming. However, these tools may also provide aggregate information programmatically on children’s progress in the five developmental domains, in order to benchmark developmental growth. To do so, it would be important to devise a common metric so that children on the various tests can be compared. Further, the common metric would be norm-referenced, allowing for comparing groups of children of different ages with their peers.

This project will assess the feasibility of developing a statistical methodology to answer the question of what are the baseline skills, knowledge and abilities of children entering preschool and kindergarten, using descriptive analytic methods. To do this, the project is focused on determining the commonalities among Teaching Strategies GOLD, High Scope Child Observation Record, and Work Sampling System assessments tools in measuring five key developmental domains.

This memorandum is the first of three and represents early analytic work examining the three tests conceptually and empirically. In this memorandum, we cover the following key areas:

1. Determine Alignment of Items and Developmental Domains across the Assessments 2. Explore Distributions for Test Developer Domain Score Distributions 3. Describe Internal Consistency Reliability of Test Developer Domain Scores4. Assess Developmental Differences and Progression by Ages and Quartiles, by test


The purpose of these tasks is to explore how each test defines key domains of development1, and, based on data collected by MA EEC, determining the distributions of the domain or sub-domain scores. The first task focuses on determining the extent to which the items and domains or sub-domains of development from each of the three assessments are aligned with each other, and how they correspond to the five larger domains of development typically considered as key areas of young children’s early skills. The second task is designed to understand the distribution of sub-domain scores for each test and whether the scores meet the assumptions of normality required for more extensive factor analyses. The third task is to describe how well the items from each test are highly correlated as intended to measure the test developer’s domains or sub-domains of development. That is, we will report on the internal consistency form of reliability for each of the tests key sub-scales. The final task is to determine whether scores from the three tests show the expected progression across ages and across groups of children at the lower and higher ends of the distribution. In this task, we would expect that a test with good measurement properties would be able to distinguish younger from older children as well as children within age groups who are operating below or above their peer group.

All analyses were conducted using those developmental domains or sub-domains designed by the test developer, to determine whether scores on these domains reflect meaningful distinctions in children’s abilities and development. The results will help to determine the feasibility of conducting the exploratory and confirmatory factor analyses in the next phases of this project.

Analytic Methods and Procedures

The assessment instruments referred to in this report include the following: Teaching Strategies Gold (GOLD), the Child Observation Record (COR), and Work Sampling System (WSS). The WSS includes five separate but related assessments: Preschool 3 (P3), Preschool 4 (P4), Head Start 3 (HS3), Head Start 4 (HS4), and Kindergarten (K).

The Work Sampling System (WSS), Teaching Strategies GOLD (GOLD), and Child Observation Record (COR) use ordinal rating scales to measure children’s progress. The WSS is based on a three-point scale, the COR uses a six-point scale, and GOLD uses a nine-point scale. For the Common Metric Project, the individual item ratings were summed to create raw scores for each subdomain and domain and a total score was then calculated by summing the domain scores.

To address missing data, scores were imputed based on valid cases. That is, a set of decision rules were made whereby a certain number of missing items would be tolerated and a 1 Domains of development are defined as those areas of development in which test items are grouped. Assessments typically group items into a number of domains according to the test developer’s conceptual framework. Although some tests group individual items into sub-domains, which are then grouped into larger domains, this was not consistently done. If they are done, and when appropriate, we will use the word ‘sub-domain” to denote groupings of items. We also make a distinction between the test developer’s domains of development and those generally accepted to be the five key developmental domains, in order to determine the alignment between the test items across the three tests and their expected relationship to these theoretical domains.


score could be imputed prorated based on the number of valid items. In most situations, if a given case was missing one, two or sometimes three items, a score for that sub-domain could still be generated. If more than the threshold number of missing items occurred, then that case was considered missing.

I. Alignment of Items and Developmental Domains across the Assessments

Question: To What Extent Do the Developmental Domains and Items Match Across the Three Assessments?

A. Alignment of Developmental Domains

The first phase of the project focuses on determining the extent to which the three assessments are conceptually aligned. Qualitative and quantitative methods were used to examine the domains, including reviewing the instruments themselves, studying the publisher’s technical support documentation, and counting the number of domains and items within domains. A summary of each assessment is provided below:

COR consists of 34 items organized into 6 major domains: 1) initiative, 2) social relations, 3) creative representation, 4) movement and music, 5) language and literacy, and 6) mathematics and science.

GOLD consists of 66 items organized into 9 major domains: 1) social-emotional, 2) physical, 3) language, 4) cognitive, 5) literacy, 6) mathematics, 7) science and technology, 8) social studies, and 9) the arts. A tenth domain, English language acquisition, is included to assess language skills for English Language Learners.

WSS: the P3 (49 items), P4 (55 items), and K (66 items) assessments are organized into 7 major domains: 1) personal and social development, 2) language and literacy, 3) mathematical thinking, 4) scientific thinking, 5) social studies, 6) the arts, and 7) physical development and health. The HS3 (54 items) and HS4 (59 items) assessments are organized into 10 major domains: 1) physical development and health, 2) social and emotional development, 3) approaches to learning, 4) logic and reasoning, 5) language development, 6) literacy and knowledge skills, 7) mathematics knowledge and skills, 8) science knowledge and skills, 9) creative arts expression, and 10) social studies knowledge and skills.

Table 1 summarizes the content of each of the three tests according to the domains listed by the test developer (shaded blue), and within the larger categories of the five key developmental domains drawn from the child development literature and used by the National Education Goals Panel (shaded pink).2 These five domains are as follows: 1) social-emotional, 2)

2 See Kagan, S. L., Moore, E., & Bredekamp, S. (Eds.), 1995. Reconsidering children's early development and learning: Toward common views and vocabulary. Report of the National Education Goals Panel, Goal 1 Technical Planning Group. Washington,


language and literacy, 3) cognitive and general knowledge, 4) approaches to learning, and 5) physical development and health.

The boundaries between these domains and constructs within them are somewhat artificial, as noted by other experts in the field.3 For example using vocabulary was categorized under language and literacy; however, because it is also relevant to understanding science and general knowledge, it could also have been categorized under cognitive and general knowledge.4 Thus, the five domains are presented here for heuristic purposes only as opposed to child development theory-building.5

Further, some domains, such as language and literacy, have longstanding research supporting their conceptual and operational definitions, while other domains, such as approaches to learning, are less well-defined and in some cases may overlap with cognitive, language and social domains.6 The key distinction in defining approaches to learning is to identify those behaviors that convey a child’s effort and engagement in classroom learning, particularly learning related to attention and persistence (focused, enduring, goal-directed learning), as well as competence motivation (initiative for effectiveness in learning).7 In this project, we use this definition of approaches to learning to classify test items and determine the degree to which the items match, with the understanding that some of the distinctions will be arbitrary, for all domains and perhaps especially for approaches to learning.

For the purposes of this project the above five key developmental domains are referred to as the Common Metric Domains to distinguish them from the Test Developer’s Domains.

Table 1. Common Metric Domains: Domain Recoding Scheme8

WSS (All Versions) Gold COR

Social-Emotional Common Metric Domain

Personal and Social Development Social Emotional Social Relations

Self-Control Self-Concept Interaction with Others

Regulates own Emotions and Behaviors Establishes and Sustains Positive Relationships Participates Cooperatively and Constructively in

Relating to Adults Relating to Other

Children

DC: U.S. Government Printing Office.3National Research Council. (2008). Child-Level Outcomes and Measures. In Early Childhood Assessment: Why, What, and How. Committee on Developmental Outcomes and Assessments for Young Children, C.E. Snow and S.B. Van Hemel, Editors. Board on Children, Youth, and Families, Board on Testing and Assessment, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. 4 Ibid., 58.5 Ibid., 58.6 Ibid., 597 McDermott, P. A., Rikoon, S. H., Waterman, C., & Fantuzzo, J. W. (2012). The Preschool Learning Behaviors Scale: Dimensionality and external validity in Head Start. School Psychology Review, 41, 66-81.8 Items are listed for the COR test while subdomains rather than items are listed for the GOLD and the WSS.


Table 1. Common Metric Domains: Domain Recoding Scheme


Social Problem Solving Group Situations

Resolving Interpersonal Conflict

Understanding and Expressing Feelings

Approaches to Learning Common Metric Domain

Approaches to Learning No Similar Domain9 Initiative

Initiative and Curiosity (HS)

Persistence and Attentiveness (HS)

Cooperation (HS)

Taking Care of Personal Needs

Making Choices and Plans

Solving Problems with Materials

Initiating Play

Language and Literacy Common Metric Domain

Language and Literacy Language and Literacy10 Language and Literacy

Listening/Receptive Language

Speaking/Expressive Language

Reading Writing Alphabet Knowledge

(HS) Print Concepts and

Conventions (HS) Engagement in English

Literacy Activities (ELL/HS only)

Listens to and Understands Increasingly Complex Language

Uses Language to Express Thoughts and Needs Comprehends and Responds to Books and

Other Texts Demonstrates Emergent Writing Skills Demonstrates Knowledge of the Alphabet Demonstrates Knowledge of Print and its uses Uses Appropriate Conversation and Other

Communication Skills Demonstrates Phonological Awareness English Language Acquisition Demonstrates Progress in Listening to and

Understanding English Demonstrates Progress in Speaking English

Listening to and Understanding Speech

Using Complex Patterns of Speech

Using Vocabulary Reading Writing Using Letter Names and

Sounds Demonstrating

Knowledge about Books

Showing Awareness of Sounds in Words

Cognitive and General Knowledge Common Metric Domain

Mathematical Thinking Mathematics Mathematics and Science

9 GOLD does not have the Approaches to Learning subdomain but there are individual items that fit the generally accepted definition of Approaches to Learning, as we will see from the results of the conceptual matching.10 Language and literacy are separate subdomains on GOLD; however they were combined here to facilitate comparison.




Mathematical Processes

Number and Operations

Patterns, Relationships, Functions

Geometry and Spatial Relations

Measurement Scientific Thinking Inquiry/Scientific Skills

and Method Conceptual Knowledge

of the Natural/Physical World (HS only)

Life Science (K only) Physical Science (K

only) Earth Science (K only)

Uses Number Concepts and Operations Demonstrates Knowledge of Patterns Explores and Describes Spatial Relationships and

Shapes Compares and Measures Science and Technology Uses Scientific Inquiry Skills Demonstrates Knowledge of the Characteristics

of Living Things Demonstrates Knowledge of the Physical

Properties of Objects and Materials Demonstrates Knowledge of Earth’s

Environment Uses Tools and other Technology to Perform

Tasks

Sorting Objects Counting Identifying Patterns Identifying Position

and Direction Comparing Properties Identifying Sequence,

Change, and Causality Identifying Natural

and Living Things Identifying Materials

and Properties

Logic and Reasoning (HS Only)

Cognitive No Similar Domain

Reasoning and Problem Solving (HS Only)

Symbolic Representation (HS Only)

Demonstrates Positive Approaches to Learning Remembers and Connects Experiences Uses Classification Skills Uses Symbols and Images to Represent

Something Not Present

Social Studies Social Studies No Similar Domain

People, Past and Present

People and Where they Live/Environment

Self, Family and Community (HS Only)

Human Interdependence

Citizenship and Government

Demonstrates Knowledge About Self Shows Basic Understanding about People and

How they Live Explores Change Related to Familiar People or

Places Demonstrates Simple Geographic Knowledge




The Arts The Arts Creative Representation

Expression and Representation (includes music, dance, art, drama)

Understanding and Appreciation

Explores the Visual Arts Explores Musical Concepts and Expression Explores Dance and Movement Concepts Explores Drama Through Actions and Language

Making and Building Models

Drawing and Painting Pictures

Pretending

Physical Development and Health Common Metric Domain

Gross Motor Development

Fine Motor Development

Personal Health and Safety

Demonstrates Gross Motor Manipulative Skills Demonstrates Fine Motor Strength Demonstrates Traveling Skills Demonstrates Balancing Skills

Moving in Various Ways

Moving with Objects Feeling and Expressing

a Steady Beat Moving to Music Singing

This table shows that, in general, the test developer’s domains across all three assessment tools are comparable, and as we can see from the items within each of these domains, there appears to be good coverage of similar items across all three tests. Further, when fitting the Test Developer Domains to the five key developmental domains there also appears to be reasonably good fit, with a few exceptions, such as the GOLD not having a subdomain corresponding to Approaches to Learning, as noted in Table 1. Minor inconsistencies were due primarily to assessments that combined domains, for example, language and literacy were treated as separate domains in GOLD, but were combined into one domain in COR. These analyses suggest that the three instruments appear to be aligned with regard to the developmental domains being assessed.

B. Alignment of Items11

Using the same methods described above, individual items from each test were examined. The goal of this analysis was to determine the degree to which the items matched in their content. The criteria for determining a match was based on the item’s developmental objective,

11 The authors would like to acknowledge the contribution of Dr. Charles Bruner, Director of the Child and Family Policy Center, for his guidance in the alignment of relevant GOLD items within the approaches to learning domain.


purpose and domain. Some items were determined to have the same developmental objective although they were worded differently, for example, “moves with some balance and control” from the WSS and “moving in various ways” from the COR were both coded as having the same objective (assessing gross motor skills) and domain (physical development and health). A match was defined as an item or objective that was included on all three instruments (COR, GOLD, and WSS).12 If an item or objective did not appear on all three instruments, it was not counted as a match. This is a very stringent requirement and thus serves as a conservative measure of the degree to which items likely correspond.13

There were a total of 407 items across the three assessment instruments (COR: n=34, GOLD: n=90), WSS: n=283): Of the total 407 items, 83 percent (n=339) matched across all the assessments instruments, while only 17 percent (n=68) were non-matches, including partial matches (i.e. matched on two of the three assessments). Of the 339 matching items, nearly 34 percent (n=115) were in cognitive domain, followed by 33 percent (n=111) language and literacy, 14 percent (n=48) physical development and health, 11 percent (n=28) social-emotional, and 8 percent (n=28) approaches to learning. Figure 1 presents the percentage of total matched items by domain:

When the five domains were examined individually, items matched across all three assessments in three of the five domains: physical health and development, language and literacy, and approaches to learning.14 In the remaining two domains, the majority of items matched across all three assessments: 71 percent of items in cognitive and general knowledge matched across all three assessments, followed by 65 percent in the social-emotional domain. Thus, complete item matches across all three assessments were found in three of the five

12 While some differences exist between the five WSS instruments, particularly between the preschool and Head Start instruments, they will be treated as a single instrument for the purposes of simplifying the findings presented in this report. 13 It may be argued that if two of the three tests had similar items, then this could be sufficient to determine a match, but this was not applied here because the goal was to see how well all three tests matched in item content.14 These analyses incorporate the definition of approaches to learning presented earlier, based on the work of Paul McDermott. Based on this definition, the creative arts items were conceptually aligned to the cognitive and general knowledge domains.


domains, with 65 percent to 71 percent of items matching in the remaining two domains. See Appendix A in this memo for a more detailed table showing matches and non-matches by assessment instrument. Figure 2 presents a summary of the matching and non-matching items by domain for the 407 total items.

In sum, these preliminary findings are suggestive of a moderate-to-high degree of alignment across the assessments items and support the plan to move forward with more in-depth analyses.

The next two sections of this memorandum will outline two relevant data quality issues that will have implications for later data analyses, primarily those starting with Phase Two of the project.

II. Exploring Distributions for Test Developer Domain Score Distributions

All test developer domain scores were standardized and then transformed to T-Scores (i.e. mean of 50 and standard deviation of 10) to facilitate interpretation. T-scores show that, for a given subset of scores, children with a score of 50 was at the mean, whereas children with scores higher than 50 are rated higher in this developmental domain or sub-domain compared with their peers and children with scores less than 50 were rated lower in this developmental domain. T-scores were based on all children’s valid scores for a given developmental domain or sub-domain.15 Additionally, the T-scores were then grouped by quartiles, so that groups of children who were rated at the lowest quarter of the sample could be compared to children in the other groups, to determine whether the mean T-scores are significantly different. By comparing quartiles and age groups of children (three-, four- and five-year olds), we can determine whether

15 T-scores were derived across all age groups of children, in order to determine whether the entire range of children’s abilities in a given domain is represented. Typically, standardized test scores are derived based on key age groups of children, to understand where a given child is located relative to the same age groups. However, in this early phase if the project we were more interested in looking at the underlying distribution across age groups in T-scores.


there is an expected developmental progression in scores within and across developmental domains and sub-domain.

Tables 2 through 5 display the descriptive statistics for each of the three assessments, showing average T-scores for each of the test developer’s domains. Since the WSS Head Start version has different domains from the WSS Preschool, they are displayed in different tables.

Table 2. Work Sampling System Preschool (Ages 3, 4, 5) Descriptive Statistics by DomainN M SD Med Skewness1 Normality Alpha

I. Personal and Social Development3

1,653 50.0 10.0 48.7 -.102 .950

II. Language and Literacy4 1,155 50.0 10.0 48.2 -.089 .952III. Mathematical Thinking5 829 50.0 10.0 45.3 .096 .936IV. Scientific Thinking6 1,032 50.0 10.0 45.4 .174 .936V. Social Studies7 800 50.0 10.0 46.3 .156 .938VI. The Arts8 901 50.0 10.0 47.5 -.147 .929VII. Physical Development & Health9

930 50.0 10.0 46.8 -.004 .952

Total Score10 1,660 50.0 10.0 48.2 -.547 .9301Skewness close to 0.0 suggests a distribution that approaches normality. 2 Mean Cronbach’s alpha for P3, P4, and K 3Number of items: P3=13, P4=13, K=13; 4Number of items: P3=10, P4=12, K=12; 5Number of items: P3=7, P4=8, K=13; 6Number of items: P3=3, P4=3, K=7; 7Number of items: P3=5, P4=8, K=10; 8Number of items: P3=4, P4=4, K=4; 9Number of items: P3=7, P4=7, K=7; 10Number of items: P3=49, P=55, K=66.

Table 3. Work Sampling System Head Start (Ages 3, 4) Descriptive Statistics by DomainN M SD Med Skewness1 Normality Alpha

I. Physical Development & Health3

1,148 50.0 10.0 44.3 .026 .950

II. Social and Emotional Development4

1,148 50.0 10.0 46.6 -.027 .941

III. Approaches to Learning5 1,149 50.0 10.0 48.1 -.021 .935IV. Logic and Reasoning6 1,142 50.0 10.0 51.1 .147 .930V. Language Development7 798 50.0 10.0 51.8 -.523 .903VI. Literacy Knowledge & Skills8

1,145 50.0 10.0 51.0 .141 .925

VII. Mathematics Knowledge & Skills9

1,145 50.0 10.0 49.4 .327 .940

VIII. Science Knowledge & Skills10

1,149 50.0 10.0 47.8 .270 .952

IX. Creative Arts Expression11

1,146 50.0 10.0 46.2 .030 .929

X. Social Studies Knowledge & Skills12

1,140 50.0 10.0 50.5 .217 .940

Total Score13 1,151 50.0 10.0 48.5 .155 .9531Skewness close to 0.0 suggests a distribution that approaches normality. 2 Mean Cronbach’s alpha for HS3 and HS4. 3Number of items: HS3=7, HS4=7; 4Number of items: HS3=9, HS4=9; 5Number of items: HS3=4, HS4=4; 6Number of items: HS3=2, HS4=3; 7Number of items: HS3=7, HS4=7; 8Number of items: HS3=6, HS4=7; 9Number of items: HS3=7, HS4=8; 10Number of items: HS3=2, HS4=2; 11Number of items: HS3=5, HS4=5; 12 Number of items: HS3=5, HS4=7; 13 Number of items: HS3=56, HS4=59.

Table 4. Child Observation Record (COR) Descriptive Statistics by DomainN M SD Med Skewness1 Normality Alpha


D1. Initiative (4 items) 258 50.0 10.0 50.1 -.123 .890D2. Social Relations (4 items)

261 50.0 10.0 50.1 -.272 .895

D3. Creative Representation (3 items)

290 50.0 10.0 52.1 -.495 .794

D4. Movement and Music (5 items)

245 50.0 10.0 51.2 -.254 .881

D5. Language and Literacy (8 items)

244 50.0 10.0 50.1 -.165 .910

D6. Mathematics and Science (8 items)

249 50.0 10.0 51.7 -.251 .947

Total Score (32 items) 328 50.0 10.0 51.4 -.349 .9691Skewness close to 0.0 suggests a distribution that approaches normality.

Table 5. Teaching Strategies GOLD Descriptive Statistics by DomainN M SD Med Skewness1 Normality Alpha

Social-Emotional (9 items) 13,979 50.0 10.0 50.0 -0.051 .970Physical (5 items) 14,148 50.0 10.0 51.9 -0.376 .963Language (8 items) 13,774 50.0 10.0 50.7 -0.276 .975Cognitive (10 items) 13,296 50.0 10.0 49.9 -0.004 .980Literacy (12 items) 10,617 50.0 10.0 49.1 0.608 .968Mathematics (7 items) 11,751 50.0 10.0 50.2 0.229 .966Science and Technology (5 items) 9,257

50.0 10.045.1 1.808

.961

Social Studies (4 items) 9,514 50.0 10.0 44.5 1.565 .951The Arts (4 items) 11,068 50.0 10.0 44.3 1.374 .962English Language Acquisition (2 items) 2,554

50.0 10.049.4 -0.078

.981

Total Score (66 items) 14,764 50.0 10.0 49.3 0.482 .9241Skewness close to 0.0 suggests a distribution that approaches normality.

The descriptive statistics display the number of valid cases for each domain, within each test. We can see that the number of valid cases varied by domain, particularly for the WSS Preschool versions. This is expected because even within the same instrument, the data are not without some missing items for each subdomain for a large number of children. As a result, subdomain scores were imputed if they only had a few items missing, described later in the “Missing Cases and Data Imputation” section. Thus, in all analyses of subdomain and total scores for each instrument, the sample sizes will vary for each subdomain within a given test, in order to maximize the available information.

The tables also show central tendency and variation, and because T-scores were calculated, they will all have the same means and standard deviations. However, the extent to which the median is similar to the mean is suggestive of whether the scores are normally distributed, and this is further assessed by the measures of skewness. A check mark for normality


indicates whether, based on skewness, that the distributions of the domain scores approximate the normal distribution.

III. Internal Consistency Reliability of Test Developer Domain Scores

Finally, Tables 2 through 5 display the total Alpha coefficients for each of the domains. For the WSS the Alpha’s are extremely high, and in a narrow range from a low of 0.93 to a high of 0.95. The internal consistencies of the COR domains were somewhat lower with a wider range, from a low of 0.79 (Creative Representation) to 0.95 (Mathematics and Science). However, it should be noted that many of the COR domains consisted of three or four items. With so few items, one would expect lower levels of reliability, whereas, for the WSS, most of the domains were comprised of 10-12 items (with some notable exceptions). Reliability for the domains on the Teaching Strategies GOLD assessment was also very high, ranging from 0.92 to 0.98. Overall, these analyses suggest that the items that were used to measure a given domain were highly consistent and suggest that they are all tapping the same underlying construct.

The normality of the domain scores suggest that the items that make up the domains of each test appear to distinguish between differences in children’s true ability distribution on the relevant developmental area. If the tests are able to measure children at different levels of ability, then these ability distributions should be shaped as normal distributions, with many children within one standard deviation above and below the mean while many fewer children are placed at each of the tails of the distribution. We expect test scores to be normally distributed. If the distribution of scores is not normal, then the test may be too hard or too easy for the range of children, and may not properly identify children’s true level of development, making it difficult to determine a common metric. Thus, the descriptive statistics for the domain scores suggest that the scores are normally distributed.

However, the assumption of normality may not be entirely correct. When viewing histograms shown in Figure 3, below for the Work Sampling System, it was apparent that large groups of children have the same T-scores and that there was a sizeable group of children with the highest T-score possible for the domain (shown by red circles). These distributions suggest that the tests may not measure correspond to the normal distribution curve, which is superimposed against the number of cases for each T-score as a point of reference. Further, the large number of children at the top end of the distribution suggests that many children may have “maxed out” the ratings, indicative of a “ceiling effect.” Instead of forming a relatively smooth and normally distributed curve suggestive of an ability distribution, we have some scores that occur at higher frequencies. This would reflect more a stepped rather than continuous curve of scores across the distribution and suggests that the tests may not be able to identify children at different levels of the ability spectrum for a given developmental domain.


Figure 3. Distributions of Work Sampling System Domain Scores.


The above histograms show potential violations of normality for the WSS assessment. Below in Figure 4, we see the distribution of domain scores on the GOLD assessment.


Figure 4. Distributions of GOLD Domain Scores.


The distributions of scores on the GOLD sub-domains suggest that many appear to meet the assumptions of normality. The sub-domains of Social-Emotional (9 items), Physical (5 items), Language (8 items), Cognitive (10 items), Literacy (12 items), Mathematics (7 items) and English Language Acquisition (2 items) show, for the most part a normal distribution. The distributions of some scores, for example, Literacy and Mathematics, are skewed to the left, suggesting that more children are rated at the lower end of scores I is also noteworthy that English Language Acquisition shows a relatively normal distribution despite comprising only two items.

However, the relatively high, narrow amplitudes of the normal curves indicate that few children scored at the tails of the distribution in most of the GOLD domains. This suggests that the test may do a good job of discriminating between children’s abilities at the center of the distribution but that there may be insufficient items at either the low end or the high end of the distribution to properly test children with significantly lower or higher abilities. Further, the following sub-domains show almost no variation and the distributions do not appear to be normal: Science and Technology (5 items), Social Studies (4 items), and The Arts (4 items). On


these items the large majority of children, upwards of 70 percent, were rated at the low end of the distribution. Also for these sub-domains, not a single high score corresponding to a rating of 3/3 was recorded for any child. On these sub-domains, children received raw scores of only 1 or 2. Thus, it would be difficult to consider these four domains as adequately distinguishing ability levels of children on these developmental areas (Science and Technology, Social Studies, and The Arts.

The next set of histograms (Figure 5) displays the distributions of domain scores for the COR assessment.


Figure 5. Distributions of COR Domain Scores.


In Figure 5, we see that many of the COR domains correspond approximately to a normal distribution, particularly Creative Representation (3 items), Language and Literacy (8 items), and Mathematics and Science (8 items). Several of these domains still display the higher amplitude whereby most children are clustered at the center of the distribution. For the remaining three domains - Initiative (4 items), Social Relations (4 items), and Movement and Music (5 items) – scores correspond to a normal distribution except for a high number of cases at the high end of the distribution, similar to what was found with the WSS domains. These two “tails” are highlighted by red circles. It is especially noteworthy for the Social Relations domain that the largest percentage of all children were rated at the two upper T-scores, suggesting that many children hit the “ceiling” in terms of their skills on this domain. The same effect, to a lesser extent, was found for the Initiative and Movement and Music domains. Overall, this suggests that the full range of children’s abilities may not be fully represented by these test domains and that we must be careful about potential violations of normality when moving forward with the factor analyses for the second phase of this project.

V. Developmental Differences Test Developer Domains, by Ages and Quartiles

An important characteristic of developmental assessments is their ability to distinguish between different age groups of children, and between different skill levels within each age group. In order for a criterion-referenced assessment to be properly scaled, we would expect that children of different ages score in a predictable manner, with an age-related progression in average scores. Younger children should score lower on a test than will older children (assuming the test is designed to cover these age groups).

Further, if we split the scores for a given developmental domain into four distinct groups according to their T-scores, in which children are placed within quartiles based on the total or subdomain scores. The quartile groups are indicative of children with different levels of abilities; those in the lowest 25 percent of cases have lower levels of skills relative to their peers in the three upper quartiles, and children with scores in the highest 25 percent are rated as being more advanced on the underlying developmental domain compared to their peers.

Statistically significant differences by age group for a given subdomain within each instrument should show a linear progression, in order to distinguish maturational differences. Statistically significant cohort differences for a given subdomain identify children within age groups at different skill levels. To show that a test does a good job of distinguishing maturation (age) differences as well as skill level (quartile) differences, we would expect there to be main effects of age and quartile in the T-scores, but that there should not be significant interaction effects. That is, an interaction between age group and quartile would suggest that the younger or older children do not hold their positions across skill levels, in which younger children should be more likely represented within the lower quartiles of scores whereas older children should be represented within the higher quartiles of scores. Older children should show age-related changes compared to younger children because, as skills develop and become more highly


integrated, children at higher skill levels show greater growth and development, compared with younger children. In order to explore age and quartile differences, a series of two-way ANOVAs were conducted with age and quartile group as the two independent variables and T-scores for each subdomain, for each instrument, were the dependent variables.

The results of these analyses are summarized briefly here, with supporting tables and figures available in Appendix B.

Table 6 summarizes the results of the two-way ANOVAs for the Work Sampling System, Preschool and Head Start versions. For the Preschool Version, only two subdomains revealed significant main effects of age; Language and Literacy and Social Studies. For these two subdomains, as would be expected, T-scores for the three year olds were significantly lower than scores for four year olds, which were, in turn, lower than scores for five year olds. However, for the total WSS score as well as five of the other domains – Personal and Social Development, Mathematical Thinking, Scientific Thinking, The Arts, and Physical Development and Health, no significant main effect was found for age. Thus, scores for the subdomains and the total score for the WSS do not consistently show the expected differences in maturation.

The WSS-Preschool instrument did reveal the expected significant quartile differences for all of the subdomain and total scores, suggesting that subgrouping of scores correspond to meaningful differences in ability levels.


Table 6. Summary of Age and Quartile Differences, Work Sampling System Total and Subdomain Scores.

Assessment and DomainsAge

DifferencesQuartile

DifferencesAge X

QuartileWork Sampling System Preschool Version

WSS-Personal and Social Development ns Q1 < Q2 < Q3 < Q4 ns

WSS Language and Literacy 3 < 4 < 5 Q1 < Q2 < Q3 < Q4 ns

WSS-Mathematical Thinking ns Q1 < Q2 < Q3 < Q4 p<.05

WSS-Scientific Thinking ns Q1 < Q2 < Q3 < Q4 p<.05

WSS-Social Studies 3 < 4 < 5 Q1 < Q2 < Q3 < Q4 p<.05

WSS-The Arts ns Q1 < Q2 < Q3 < Q4 ns

WSS-Physical Development and Health ns Q1 < Q2 < Q3 < Q4 p<.05

WSS-Total Score ns Q1 < Q2 < Q3 < Q4 ns

Work Sampling System Head Start Version16

WSSHS Physical Development and Health 3 < 4 Q1 < Q2 < Q3 < Q4 p<.05

WSSHS Social and Emotional Development 3 < 4 Q1 < Q2 < Q3 < Q4 ns

WSSHS Approaches to Learning ns Q1 < Q2 < Q3 < Q4 ns

WSSHS Logic and Reasoning 3 < 4 Q1 < Q2 < Q3 < Q4 p<.05

WSSHS Language Development 3 < 4 Q1 < Q2 < Q3 < Q4 p<.05

WSSHS Literacy Knowledge and Skills 3 < 4 Q1 < Q2 < Q3 < Q4 p<.05

WSSHS Mathematics Knowledge and Skills 3 < 4 Q1 < Q2 < Q3 < Q4 p<.05

WSSHS Science Knowledge and Skills ns Q1 < Q2 < Q3 < Q4 ns

WSSHS Creative Arts Expression 3 < 4 Q1 < Q2 < Q3 < Q4 nsWSSHS Social Studies Knowledge and Skills 3 < 4 Q1 < Q2 < Q3 < Q4 ns

WSSHS Total Score 3 < 4 Q1 < Q2 < Q3 < Q4 p<.05

However, there was also a number of significant age by quartile interaction effects. Through post-hoc comparisons of group means, these interaction effects indicate that the age groups do not always maintain the developmental progression across all ability groups (cohort). Ideally, we would not expect to find interaction effects, so that age groups are parallel across quartiles, with little intersection. Figure 6 illustrates these differences (additional figures are displayed in Appendix B).

16 There were no data for five-year olds. Thus, the comparisons were between the three- and four-year olds.


Q1 Q2 Q3 Q425

35

45

55

65

Figure 6. WSS-Preschool Social Studies Subdomain, Age by Quartile Effects

Three-Year OldsFour-Year OldsFive-Year Olds

Quartile

Aver

age

T-Sc

ore

As we can see in Figure 6, three-year old children who were in the lowest quartile (Q1), had lower scores compared to four year olds, but five year olds in the lowest quartile also had scores similar to three-year olds. We would expect the five year olds to have higher scores even at the lowest ability levels. As we move from the first to the second quartile, we find that the five year olds have scores similar to the four-year olds and the three year olds in Q2 have the lowest scores. But at the third quartile, the five years show significantly higher scores while the three- and four-year olds have similar scores. But at the highest quartile, the five year old no longer hold this advantage over the other children and in fact show significantly lower scores compared to the three- and four-year olds. Thus, there seems little correspondence between ability levels as represented by the quartiles and children of different ages. As noted earlier and shown in Table 6, there were significant interaction effects for other sub-domains of the WSS-Preschool.

Regarding scores on the Work Sampling System Head Start Version, there were significant age group differences on almost all subdomains, with the exception of Approaches to Learning and Science Knowledge and Skills. These differences were in the expected direction with the four-year olds showing higher T-scores compared with three-year olds. There were also significant quartile differences in the expected direction, but there were also a number of significant age by quartile interaction effects, suggesting that the scores do not align according age and skill level differences in the expected direction (Appendix B shows tables with the means as well as figures for the interaction effects). The interaction effects for the total WSS-Head Start score show the general pattern in these results (Figure 7). Whereas three-year olds scored lower than four-year olds, as expected, at the bottom quartile (Q1) by the second quartile (Q2) the gap in scores between three- and four-year olds narrowed so that they were not significantly different. At the highest ability level (Q4) there were also non-significant differences, suggesting that the test does not distinguish between children at different developmental periods.

Table 7 shows the results of the two-way ANOVAs for the Child Observation Record. For this instrument, data were available only for three and four year olds.


Table 7. Summary of Age and Quartile Differences for Child Observation Record (COR) Total and Subdomain Scores.

Assessment and Domains Age Differences

Quartile Differences AgeXQuartile

COR Initiative 3 < 4 Q1 < Q2 < Q3 < Q4 ns

COR Social Relations ns Q1 < Q2 < Q3 < Q4 ns

COR Creative Representation 3 < 4 Q1 < Q2 < Q3 < Q4 ns

COR Movement and Music 3 < 4 Q1 < Q2 < Q3 < Q4 p<.05

COR Language and Literacy ns Q1 < Q2 < Q3 < Q4 ns

COR Mathematics and Science 3 < 4 Q1 < Q2 < Q3 < Q4 ns

COR Total Score 3 < 4 Q1 < Q2 < Q3 < Q4 p<.05

As we can see in Table 7, there were significant age and cohort differences for many of the subdomain scores as well as the total score, although there were no differences between three- and four-year olds for the subdomains of Social Relations and Language and Literacy. However, there were relatively few interaction effects, with the exception of the Movement and Music subdomain and the COR total score. Figure 8 illustrates the interaction effects in the COR.

COR total scores were lowest for three-year olds, compared to four-year olds, at the lowest quartile, as expected, and these differences narrowed in the two middle quartiles, and, at the highest quartile, the initial differences between three- and four-year olds were evident but were much smaller than those among children at the lowest quartile. These interaction effects suggest that, at the middle ability levels, there are no significant differences between the three-

Q1 Q2 Q3 Q425.0

35.0

45.0

55.0

65.0

Figure 8. Child Observation Record (COR) Total Score, Age by Quartile Effects

Three-Year Olds

Four-Year Olds

Quartile

Aver

age

T-Sc

ore

Q1 Q2 Q3 Q425.0

35.0

45.0

55.0

65.0

Figure 7. WSS-Head Start Total Score, Age by Quartile Effects

Three-Year OldsFour-Year Olds

Quartile

Aver

age

T-Sc

ore


and four-year olds. Overall, the COR total score appears to do a better job of distinguishing age differences among those only at the lowest skill levels.

Table 8 summarizes the results of the two-way analyses for the Teaching Strategies’ GOLD total and domain scores.17 As we can see, there were significant main effects of age for all subdomain scores, although for the Physical domain the one-year olds had lower scores than the two- and three-year olds, who were not significantly different from each other. These two age groups (2-3 year olds) were significantly different from the four- and five-year olds, who were also not significantly different from each other. For all domains, there were significant quartile differences as expected, but there were also significant age by quartile group interaction effects. These interaction effects are illustrated in Figure 9.

17 Four domains (English Acquisition, Science and Technology, Social Studies, and the Arts were excluded from these analyses due to small cell sizes and the previously discussed issues related to normality and variability.


Table 8. Summary of Age and Quartile Differences, Teaching Strategies GOLD Total and Subdomain Scores.

Assessment and DomainsAge

DifferencesQuartile

DifferencesAge X Quartile

Social Emotional 1<2<3<4<5 Q1 < Q2 < Q3 < Q4 p<.05

Physical 1<2,3<4,5 Q1 < Q2 < Q3 < Q4 p<.05

Language 1<2<3<4<5 Q1 < Q2 < Q3 < Q4 p<.05

Cognitive 1<2<3<4<5 Q1 < Q2 < Q3 < Q4 p<.05

Literacy 1<2<3<4<5 Q1 < Q2 < Q3 < Q4 p<.05

Mathematics 1<2<3<4<5 Q1 < Q2 < Q3 < Q4 p<.05

In Figure 9, the one-year olds in the lowest quartile were rated lower than the other children, as expected, although the differences between the children in the other age groups was not significant for children scoring in the lowest quartile. At the two middle quartiles, the age differences are not significant although the one-year olds actually had higher scores than their older peers at these ability levels. At the highest quartile (Q4), the one-year olds were significantly lower than the other ages, although the order of scores did not follow the expected developmental trend by age. While the four- and five-year olds in the highest quartile had significantly higher scores than the other age groups, the three-year old scores were lower than the two-year old scores. Thus, while some age differences within quartiles corresponded to expected differences, the scores on the Literacy domain do not show a consistent trend across the quartiles and, in the two middle quartiles, the five age groups are not highly distinguishable.

Q1 Q2 Q3 Q430.0

40.0

50.0

60.0

Figure 9. Teaching Strategies GOLD Literacy Domain Score, Age by Quartile Ef -fects

One-Year OldsTwo-Year OldsThree-Year OldsFour-Year OldsFive-Year Olds

Quartile

Aver

age

T-Sc

ore


These interaction effects suggest that the GOLD scores in general do not correspond to significant differences in age and ability levels.

VI. Data Quality and Missing Data

Question: Is the Quality of the Data Provided Sufficient for the Planned Analyses?

A. Data Collection Time Points

According to the publisher guidelines, each of the three assessments should include at least three periods of data collection to assess child progress. In the data files supplied by MA EEC, the data entry time points associated with these periods are labeled as follows: P1 (Summer 2011/2012), P2 (Fall 2012/2013), P3 (Winter 2012/2013), and P4 (Spring 2012/2013).

Based on the files provided in November of 2012, only the P1 time point (Summer 2011-2012) contained data from all three instruments. Of the three assessment data files, only the GOLD file contained data for all three time points. (Note: the GOLD P3 data file was truncated and included scores for the social emotional domain only). Table 9 summarizes the data time points represented by the data files provided in November, 2012.

Table 9. Common Metric Data by Time Point for Data files sent January, 2012a

Instrument Spring 2011-2012 Summer 2011- 2012 (P1)

Fall 2012-2013 (P2)

Winter 2012-2013 (P3)

COR No Data Yes No Data No Data

GOLD Yes Yes Yes No Data

WSS No Data Yes No Data No Dataa. Includes all age groups.

In order to assess developmental change over time, we will need data from more than one time point and ideally for three time points (according to the test publishers) for each of the assessments. Further, when examining data from additional time points, it would be necessary to make sure that the assessment periods are consistent and do not overlap. For example, the data collection time periods and data entry deadlines for the fall and winter should be identified as distinct time periods. Nevertheless, the current dataset suggests that it would be impossible to measure change over time given the lack of additional data points.

B. Missing Data

In general, decisions about how to deal with missing data must take into account the amount of missing data, the type of missing data, and the reasons for the missing data. In examining the data files for the three assessments, there were many cases where only a few items


were missing, probably due to random errors completing all the ratings and inputting the data into the online database. It was determined that these “missing at random” cases can be “recoverable.” The approach we used was a simple one: a) Assess each subdomain of an instrument separately for cases with missing items and then b) Apply a set of rules for determining the maximum of missing items given the number of possible items and prorate the subdomain score based on the number of valid items.

For subdomains with three to six items, cases were assigned subdomain scores based on the number of valid items if no more than 1 item was missing. For subdomains with seven to fourteen items, cases were assigned valid subdomain scores if no more than 2 items were missing; and for subdomains with more than 15 items, cases were assigned subdomain scores if no more than 3 items were missing.

For cases that met these criteria for each subdomain of a test, scores were derived based on the prorated raw scores from the valid items for a given subdomain. The argument for including up to three items with missing scores (for subdomains with more than 15 items) as part of a complete dataset was based on the likelihood that this minimal number of missing items can allow a valid score to be estimated from the remaining data without greatly affecting the subdomain average scores. This also assumes that the scores were missing at random and were not missing due to systematic reasons. The percentages of cases for each instrument that were imputed varied from 0.2 percent for the WSS Head Start instrument, to 3.5 percent for the COR, 3.8 percent for the WSS Preschool instrument, and 7 percent for Teaching Strategies GOLD.18

Even with this relatively modest imputation method, there were substantial numbers of missing cases for each instrument. Table 10 summarizes the number of valid and missing data by domain for each of the three assessments after the above imputation occurred. For all of the tests with the exception of the WSS-HS, overall there is a very high percentage of missing data across all domains.

By way of comparison, across the three instruments, MA EEC provided NYU with an earlier data set comprising a total of 9,582 cases from the fall and winter of 2011-12. Upon initial examination of the data, NYU reported that 35 percent of fall cases and 29 percent of winter cases were immediately removed from further analysis because they contained only child demographic information, but no assessment or score data. The amount of missing data in the current dataset is similar to this level with a higher percentage of missing cases notably for the Teaching Strategies GOLD instrument and slightly lower levels for the other instruments.

18 These percentages were calculated as the harmonic means for the average percentage of imputed scores for all subdomain scores within an instrument, because valid and missing cases were not independent for subdomains. That is, a given case could have missing items in more than subdomain score and thus the average percentage imputed had to take into account the “weight” of missing cases for each subdomain.


Table 10. Valid and Missing Cases by Assessment and DomainValid N Missing

NTotal N Percent

MissingWork Sampling System Preschool (WSS)I. Personal and Social Development 1,155 564 2,217 25.4%II. Language and Literacy 829 1,062 2,217 47.9%III. Mathematical Thinking 1,032 1,388 2,217 62.6%IV. Scientific Thinking 800 1,185 2,217 53.5%V. Social Studies 901 1,417 2,217 63.9%VI. The Arts 930 1,316 2,217 59.4%VII. Physical Development & Health 1,660 1,287 2,217 58.1%Total Score 1,155 557 2,217 25.1%Work Sampling System Head Start (WSS)I. Physical Development & Health 1,148 6 1,154 0.5%II. Social and Emotional Development 1,148 6 1,154 0.5%III. Approaches to Learning 1,149 5 1,154 0.4%IV. Logic and Reasoning 1,142 12 1,154 1.0%V. Language Development 798 356 1,154 30.8%VI. Literacy Knowledge & Skills 1,145 9 1,154 0.8%VII. Mathematics Knowledge & Skills 1,145 9 1,154 0.8%VIII. Science Knowledge & Skills 1,149 5 1,154 0.4%IX. Creative Arts Expression 1,146 8 1,154 0.7%X. Social Studies Knowledge & Skills 1,140 14 1,154 1.2%Total Score 1,151 3 1,154 0.3%Teaching Strategies GOLDSocial-Emotional 13,979 13,603 27,582 49.3%Physical 14,148 13,434 27,582 48.7%Language 13,774 13,808 27,582 50.1%Cognitive 13,296 14,286 27,582 51.8%Literacy 10,617 16,965 27,582 61.5%Mathematics 11,751 15,831 27,582 57.4%Science and Technology 9,257 18,325 27,582 66.4%Social Studies 9,514 18,068 27,582 65.5%The Arts 11,068 16,514 27,582 59.9%English Language Acquisition 2,554 25,028 27,582 90.7%Total Score 14,764 12,818 27,582 46.5%Child Observation Record (COR)D1. Initiative 258 197 455 43.3%D2. Social Relations 261 194 455 42.6%D3. Creative Representation 290 165 455 36.3%D4. Movement and Music 245 210 455 46.2%D5. Language and Literacy 244 211 455 46.4%D6. Mathematics and Science 249 206 455 45.3%Total Score 328 127 455 27.9%


The high number of missing data for the GOLD may be due to the participation of the Boston school district in which they only completed the Social-Emotional Domain items. Still, this would not explain the almost 50 percent of cases missing from just this single domain. It also should be noted that the NYU analyses did not look at missing data at the same level of detail that we did (by sub-domain and using the same missing data criteria).

The lack of complete data makes it difficult to determine whether the three assessment instruments can yield a common metric, in a number of ways. First, with a reduced number of cases, we cannot be sure that the full continuum of children’s abilities in a given domain is properly represented. That is, some subgroups of the population may not be represented in the data, especially those subgroups where children are at the lower or upper end of the ability distribution. As well, the potential violations of normality that we found may be due at least in part to the reduced number of valid cases, suggesting that if/when the missing data are “filled in” we might indeed find more of a normal distribution of scores. Finally, even though the number of valid cases appears large enough to override concerns about missing data, the analyses become more unstable the more that the sample is broken into different age groups and ability/skill levels. This is especially true for the two-way ANOVAs where some of the cell sizes are relatively small. It is possible that some of these analyses did not reach levels of statistical significance due to relatively low statistical power.

There are two recommendations that we can make based on this level of missing data, which far exceeds the convention for the amount of missing data “allowed” to continue the sophisticated analyses we have planned in the next phases of this project. First, the factor analytic analyses should be considered preliminary due to the amount of missing data, and we are concerned we may not have enough cases to properly explore some subgroups of the population. A second suggestion is that it would be useful to discuss these missing data issues and potential strategies for improving the quality of future assessment data.

Massachusetts EEC Common Metric Progress Memorandum 31

APPENDIX A


REVISED Appendix Table A-1. Matches by Instrument and Common Metric Domain (N=407)

Match Non-MatchTotal Items n % n %

CORLanguage/Literacy 10 10 100.0 0 0.0Cognitive 14 14 100.0 0 0.0Social Emotional 4 4 100.0 0 0.0Physical/Health 3 3 100.0 0 0.0Approaches to Learning 3 3 100.0 0 0.0Total 34 34 100.0 0 0.0

GOLDLanguage/Literacy 42 42 100.0 0 0.0Cognitive 25 19 76.0 6 24.0Social Emotional 8 8 100.0 0 0.0Physical/Health 10 10 100.0 0 0.0Approaches to Learning 5 5 100.0 0 0.0Total 90 84 93.3 6 6.7

WSS-P3Language/Literacy 10 10 100.0 0 0.0Cognitive 19 14 73.7 5 26.3Social Emotional 9 5 55.6 4 44.4Physical/Health 7 7 100.0 0 0.0Approaches to Learning 4 4 100.0 0 0.0Total 49 40 81.6 9 18.4

WSS-P4Language/Literacy 12 12 100.0 0 0.0Cognitive 23 15 65.2 8 34.8Social Emotional 9 5 55.6 4 44.4Physical/Health 7 7 100.0 0 0.0Approaches to Learning 4 4 100.0 0 0.0Total 55 43 78.2 12 21.8

WSS-HS3Language/Literacy 12 12 100.0 0 0.0Cognitive 22 17 77.3 5 22.7Social Emotional 9 5 55.6 4 44.4Physical/Health 7 7 100.0 0 0.0Approaches to Learning 4 4 100.0 0 0.0Total 54 45 83.3 9 16.7

WSS-HS4Language/Literacy 13 13 100.0 0 0.0Cognitive 26 18 69.2 8 30.8Social Emotional 9 5 55.6 4 44.4Physical/Health 7 7 100.0 0 0.0Approaches to Learning 4 4 100.0 0 0.0Total 59 47 71.2 12 20.3

WSS-KLanguage/Literacy 12 12 100.0 0 0.0Cognitive 34 18 52.9 16 47.1


Match Non-MatchTotal Items n % n %

Social Emotional 9 5 55.6 4 44.4Physical/Health 7 7 100.0 0 0.0Approaches to Learning 4 4 100.0 0 0.0Total 66 46 69.7 20 30.3

TotalLanguage/Literacy 111 111 100.0 0 0.0Cognitive 163 115 70.6 48 29.4Social Emotional 57 37 64.9 20 35.1Physical/Health 48 48 100.0 0 0.0Approaches to Learning 28 28 100.0 0 0.0Total 407 339 83.3 68 16.7


REVISED Appendix Table A-2. Item Matches

Match/Objectiv

e#Tool Developer’s Domain Common Metric Domain Item ID Item

1 COR Language and Literacy Language and Literacy Q Listening to and understanding speech.

1 COR English Language Learners Language and Literacy HH Listening and understanding (ELL).

1 GOLD Language Language and Literacy 8a Comprehends Language.

1 GOLD Language Language and Literacy 8b Follows directions.

1 GOLD English Language Learners Language and Literacy 37 Demonstrates progress in listening to and understanding English (ELL).

1 GOLD Language/English Language Learners Language and Literacy S8a Comprehends Language (ELL).

1 GOLD Language/English Language Learners Language and Literacy S8b Follows directions (ELL).

1 WSS-P3 Language and Literacy Language and Literacy II-A2 Follows two-step directions.

1 WSS-P3 Language and Literacy Language and Literacy II-A1 Gains meaning by listening.

1 WSS-P4 Language and Literacy Language and Literacy II-A2 Follows two or three-step directions.

1 WSS-P4 Language and Literacy Language and Literacy II-A1 Gains meaning by listening.

1 WSS-HS3

Language Development Language and Literacy V-A2 Follows two-step directions.

1 WSS-HS3

Language Development Language and Literacy V-A1 Gains meaning by listening.

1 WSS-HS3

English Language Learners Language and Literacy V-A3 For non-English speaking children, comprehends or understands the English language (ELL).

1 WSS-HS4

Language Development Language and Literacy V-A2 Follows two- or three-step directions.

1 WSS-HS4

Language Development Language and Literacy V-A1 Gains meaning by listening.

1 WSS-HS4

Language/English Language Learners Language and Literacy V-A3 For non-English speaking children, comprehends or understands the English language (ELL).

1 WSS-K Language and Literacy Language and Literacy II-A2 Follows directions that involve a series of actions.

1 WSS-K Language and Literacy Language and Literacy II-A1 Gains meaning by listening.


Match/Objectiv


2 COR Language and Literacy Language and Literacy R Using vocabulary.

2 GOLD Language and Literacy Language and Literacy 9a Uses and expanding expressive vocabulary.

2 GOLD Language/English Language Learners Language and Literacy S9a Uses an expanding expressive vocabulary (ELL).

2 WSS-P3 Language and Literacy Language and Literacy II-B2 Uses expanded vocabulary and language for a variety of purposes.


2 WSS-HS3

Language Development Language and Literacy V-B2 Uses expanded vocabulary and language for a variety of purposes.

2 WSS-HS4


2 WSS-K Language and Literacy Language and Literacy II-B2 Uses expanded vocabulary and language for a variety of purposes.

3 COR Language and Literacy Language and Literacy S Using complex patterns of speech.

3 COR Language/English Language Learners Language and Literacy GG Speaking and communicating (ELL).

3 GOLD Language Language and Literacy 9b Speaks clearly.

3 GOLD Language Language and Literacy 9c Uses conventional grammar.

3 GOLD Language Language and Literacy 9d Tells about another time or place.

3 GOLD Language Language and Literacy 10a Engages in conversations.

3 GOLD Language Language and Literacy 10b Uses social rules of language.

3 GOLD Language/English Language Learners Language and Literacy 38 Demonstrates progress in speaking English (ELL).

3 GOLD Language/English Language Learners Language and Literacy S9b Speaks clearly (ELL).

3 GOLD Language/English Language Learners Language and Literacy S9c Uses conventional grammar (ELL).

3 GOLD Language/English Language Learners Language and Literacy S9d Tells about another time or place (ELL).

3 GOLD Language/English Language Learners Language and Literacy S10a Engages in conversations (ELL).


Match/Objectiv


3 GOLD Language/English Language Learners Language and Literacy S10b Uses social rules of language (ELL).

3 WSS-P3 Language and Literacy Language and Literacy II-B1 Speaks clearly enough to be understood by most listeners.



3 WSS-P4 Language and Literacy Language and Literacy II-B1 Speaks clearly enough to be understood without contextual clues.

3 WSS-HS3

Language Development Language and Literacy V-B1 Speaks clearly enough to be understood by most listeners.

3 WSS-HS3


3 WSS-HS3

Language/English Language Learners Language and Literacy V-B3 For non-English speaking children, speaks or uses English (ELL).

3 WSS-HS4

Language Development Language and Literacy V-B1 Speaks clearly enough to be understood without contextual clues.

3 WSS-HS4


3 WSS-HS4

Language/English Language Learners Language and Literacy V-B3 For non-English speaking children, speaks or uses English (ELL).

3 WSS-K Language and Literacy Language and Literacy II-B1 Speaks clearly and conveys ideas effectively.

3 WSS-K Language and Literacy Language and Literacy II-B2 Uses expanded vocabulary and language for a variety of purposes.

4 COR Language and Literacy Language and Literacy T Showing awareness of sounds in word.

4 GOLD Language and Literacy Language and Literacy 15c Notices and discriminates smaller and smaller units of sound.

4 GOLD Literacy Language and Literacy 15a Notices and discriminates rhyme.

4 GOLD Literacy Language and Literacy 15b Notices and discriminates alliteration.

4 GOLD Literacy/English Language Learners Language and Literacy S15c Notices and discriminates smaller and smaller units of sound (ELL).

4 GOLD Literacy/English Language Learners Language and Literacy S15a Notices and discriminates rhyme (ELL).

4 GOLD Literacy/English Language Learners Language and Literacy S15b Notices and discriminates alliteration (ELL).


Match/Objectiv


4 WSS-P3 Language and Literacy Language and Literacy II-A3 Shows beginning phonological awareness.

4 WSS-P4 Language and Literacy Language and Literacy II-A3 Demonstrates phonological awareness.

4 WSS-HS3

Literacy Knowledge and Skills Language and Literacy VI-B1 Shows beginning phonological awareness.

4 WSS-HS4

Literacy Knowledge and Skills Language and Literacy VI-B1 Demonstrates phonological awareness.

4 WSS-K Language and Literacy Language and Literacy II-A3 Demonstrates phonological awareness.

5 COR Language and Literacy Language and Literacy U Demonstrating knowledge about books.

5 GOLD Literacy Language and Literacy 17a Uses and appreciates books.

5 GOLD Literacy Language and Literacy 17b Uses print concepts.

5 GOLD Literacy/English Language Learners Language and Literacy S17a Uses and appreciates books (ELL).

5 GOLD Literacy/English Language Learners Language and Literacy S17b Uses print concepts (ELL).

5 GOLD Literacy Language and Literacy 18a Interacts during read-alouds and book conversations

5 GOLD Literacy/English Language Learners Language and Literacy S18a Interacts during read-alouds and book conversations (ELL).

5 WSS-P3 Literacy Language and Literacy II-C3 Comprehends and responds to stories read aloud.

5 WSS-P3 Language and Literacy Language and Literacy II-C1 Shows appreciation for books.

5 WSS-P4 Language and Literacy Language and Literacy II-C4 Comprehends and responds to stories read aloud.

5 WSS-P4 Language and Literacy Language and Literacy II-C1 Shows appreciation for books and reading.

5 WSS-P4 Language and Literacy Language and Literacy II-C2 Shows beginning understanding of concepts about print.

5 WSS-HS3

Literacy Knowledge and Skills Language and Literacy VI-A1 Shows appreciation for books.

5 WSS-HS3

Literacy Knowledge and Skills Language and Literacy VI-A2 Comprehends and responds to stories read aloud.

5 WSS-HS3

Literacy/English Language Learners Language and Literacy V-C1 For non-English speaking children, understands and responds to books, storytelling, and songs presented in English (ELL).


Match/Objectiv


5 WSS-HS4

Literacy Knowledge and Skills Language and Literacy VI-A1 Shows appreciation for books and reading.

5 WSS-HS4


5 WSS-HS4

Literacy Knowledge and Skills Language and Literacy VI-D1 Shows beginning understanding of concepts about print.

5 WSS-HS4

Literacy/English Language Learners Language and Literacy V-C1 For non-English speaking children, understands and responds to books, storytelling, and songs presented in English (ELL).5 WSS-K Language and Literacy Language and Literacy II-C1 Shows interest in and knowledge about books and reading.

5 WSS-K Language and Literacy Language and Literacy II-C2 Shows some understanding of concepts about print.

6 COR Language and Literacy Language and Literacy V Using letter names and sounds.

6 GOLD Literacy Language and Literacy 16b Uses letter-sound knowledge.

6 GOLD Literacy Language and Literacy 16a Identifies and names letters.

6 GOLD Literacy/English Language Learners Language and Literacy S16b Uses letter-sound knowledge (ELL).

6 GOLD Literacy/English Language Learners Language and Literacy S16a Identifies and names letters (ELL).

6 WSS-P3 Language and Literacy Language and Literacy II-C2 Shows interest in letters and words.

6 WSS-P4 Language and Literacy Language and Literacy II-C3 Begins to develop knowledge about letters.

6 WSS-HS3

Literacy Knowledge and Skills Language and Literacy VI-C1 Shows interest in letters and words.

6 WSS-HS4

Literacy Knowledge and Skills Language and Literacy VI-C1 Begins to develop knowledge about letters.

6 WSS-K Language and Literacy Language and Literacy II-C3 Knows letters, sounds, and how they form words.

7 COR Language and Literacy Language and Literacy W Reading.

7 GOLD Literacy Language and Literacy 18b Uses emergent reading skills.

7 GOLD Literacy/English Language Learners Language and Literacy S18b Uses emergent reading skills (ELL).



Match/Objectiv



7 WSS-HS3


7 WSS-HS4


7 WSS-K Language and Literacy Language and Literacy II-C1 Shows interest in and knowledge about books and reading.

8 COR Language and Literacy Language and Literacy X Writing.

8 GOLD Literacy Language and Literacy 19b Writes to convey meaning.

8 GOLD Literacy Language and Literacy 19a Writes name.

8 GOLD Literacy/English Language Learners Language and Literacy S19b Writes to convey meaning (ELL).

8 GOLD Literacy/English Language Learners Language and Literacy S19a Writes name (ELL).

8 WSS-P3 Language and Literacy Language and Literacy II-D2 Uses scribbles and unconventional shapes to write.

8 WSS-P4 Language and Literacy Language and Literacy II-D2 Uses letter-like shapes, symbols, and letters to convey meaning.

8 WSS-P4 Language and Literacy Language and Literacy II-D3 Understands purposes for writing.

8 WSS-HS3

Literacy Knowledge and Skills Language and Literacy VI-D1 Uses scribbles and unconventional shapes to write.

8 WSS-HS4

Literacy Knowledge and Skills Language and Literacy VI-E1 Uses letter-like shapes, symbols, and letters to convey meaning

8 WSS-K Language and Literacy Language and Literacy II-D2 Uses letter-like shapes, symbols, and letters to convey meaning.

8 WSS-K Language and Literacy Language and Literacy II-D3 Understands purposes for writing.

9 GOLD Comprehends and responds to books and other texts

Language and Literacy 18c Retells stories.

9 GOLD Literacy/English Language Learners Language and Literacy S18c Retells stories (ELL).

9 COR Language and Literacy Language and Literacy U Demonstrating knowledge about books.



Match/Objectiv





9 WSS-HS3


9 WSS-HS3


9 WSS-HS4


9 WSS-HS4


9 WSS-K Language and Literacy Language and Literacy II-C1 Shows interest in and knowledge about books and reading.

9 WSS-K Language and Literacy Language and Literacy II-C4 Comprehends and responds to fiction and nonfiction text.

10 COR Science and Mathematics Cognitive and General Knowledge Y Sorting objects.

10 GOLD Cognitive Cognitive and General Knowledge 13 Uses classification skills.

10 WSS-P3 Mathematical Thinking Cognitive and General Knowledge III-C1 Sorts objects into subgroups that vary by one attribute.

10 WSS-P4 Mathematical Thinking Cognitive and General Knowledge III-C1 Sorts objects into subgroups that vary by one or two attributes.

10 WSS-HS3

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-D1 Sorts objects into subgroups that vary by one attribute.

10 WSS-HS4

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-D1 Sorts objects into subgroups that vary by one or two attributes.

10 WSS-K Mathematical Thinking Cognitive and General Knowledge III-C1 Sorts objects into subgroups classifying and comparing according to a rule.

11 COR Science and Mathematics Cognitive and General Knowledge Z Identifying patterns.

11 GOLD Mathematics Cognitive and General Knowledge 23 Demonstrates knowledge of patterns.

11 WSS-P3 Mathematical Thinking Cognitive and General Knowledge III-C1 Sorts objects into subgroups that vary by one attribute.

11 WSS-P4 Mathematical Thinking Cognitive and General Knowledge III-C1 Sorts objects into subgroups that vary by one or two attributes.


Match/Objectiv


11 WSS-P4 Mathematical Thinking Cognitive and General Knowledge III-C2 Recognizes simple patterns and duplicates them.

11 WSS-HS3

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-D1 Sorts objects into subgroups that vary by one attribute.

11 WSS-HS4

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-D1 Sorts objects into subgroups that vary by one or two attributes.

11 WSS-HS4

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-D2 Recognizes simple patters and duplicates them

11 WSS-K Mathematical Thinking Cognitive and General Knowledge III-C2 Recognizes, duplicates, and extends patterns.

11 WSS-K Mathematical Thinking Cognitive and General Knowledge III-C1 Sorts objects into subgroups classifying and comparing according to a rule.

12 COR Science and Mathematics Cognitive and General Knowledge BB Counting.

12 GOLD Mathematics Cognitive and General Knowledge 20a Counts.

12 GOLD Mathematics Cognitive and General Knowledge 20b Quantifies.

12 GOLD Mathematics Cognitive and General Knowledge 20c Connects numerals with their quantities.

12 WSS-P3 Mathematical Thinking Cognitive and General Knowledge III-B1 Shows curiosity and interest in counting and numbers.

12 WSS-P4 Mathematical Thinking Cognitive and General Knowledge III-B1 Shows beginning understanding of number and quantity.

12 WSS-HS3

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-A1 Shows curiosity and interest in counting and numbers.

12 WSS-HS4

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-A1 Shows beginning understanding of number and quantity

12 WSS-K Mathematical Thinking Cognitive and General Knowledge III-B1 Shows understanding of number and quantity.

12 WSS-K Mathematical Thinking Cognitive and General Knowledge III-B2 Begins to understand relationships between quantities

13 COR Science and Mathematics Cognitive and General Knowledge CC Identifying position and direction.

13 GOLD Mathematics Cognitive and General Knowledge 21a Understands spatial relationships.

13 GOLD Mathematics Cognitive and General Knowledge 21b Understands shapes.

13 WSS-P3 Mathematical Thinking Cognitive and General Knowledge III-D1 Identifies several shapes.


Match/Objectiv


13 WSS-P3 Mathematical Thinking Cognitive and General Knowledge III-D2 Shows understanding of several positional words.

13 WSS-P4 Mathematical Thinking Cognitive and General Knowledge III-D2 Shows understanding of and uses several positional words.

13 WSS-P4 Mathematical Thinking Cognitive and General Knowledge III-D1 Begins to recognize and describe the attributes of shapes.

13 WSS-HS3

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-C1 Identifies several shapes.

13 WSS-HS3

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-C2 Shows understanding of several positional words.

13 WSS-HS4

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-C1 Begins to recognize and describe the attributes of shapes.

13 WSS-HS4

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-C2 Shows understanding of and uses several positional words.

13 WSS-K Mathematical Thinking Cognitive and General Knowledge III-D1 Recognize and describes some attributes of shapes.

13 WSS-K Mathematical Thinking Cognitive and General Knowledge III-D2 Shows understanding of and uses direction, location, and position words.

14 COR Science and Mathematics Cognitive and General Knowledge AA Comparing properties.

14 GOLD Mathematics Cognitive and General Knowledge 22 Compares and measures.

14 GOLD Science and Technology Cognitive and General Knowledge 28 Uses tools and other technology to perform tasks.

14 WSS-P3 Mathematical Thinking Cognitive and General Knowledge IV-A3 Makes comparisons among objects.

14 WSS-P3 Mathematical Thinking Cognitive and General Knowledge III-E1 Shows understanding of some comparative words.

14 WSS-P3 Mathematical Thinking Cognitive and General Knowledge III-E2 Participates in measuring activities.

14 WSS-P3 Scientific Thinking Cognitive and General Knowledge IV-A1 Uses senses to observe and explore classroom materials and natural phenomena.

14 WSS-P3 Scientific Thinking Cognitive and General Knowledge IV-A2 Begins to use simple tools and equipment for investigation.

14 WSS-P4 Scientific Thinking Cognitive and General Knowledge IV-A3 Makes comparisons among objects.

14 WSS-P4 Scientific Thinking Cognitive and General Knowledge III-E1 Orders, compares, and describes objects according to a single attribute.

14 WSS-P4 Scientific Thinking Cognitive and General Knowledge III-E2 Participates in measuring activities.


Match/Objectiv


14 WSS-P4 Scientific Thinking Cognitive and General Knowledge IV-A1 Asks questions and uses senses to observe and explore materials and natural phenomena.

14 WSS-P4 Scientific Thinking Cognitive and General Knowledge IV-A2 Uses simple tools and equipment for investigation.

14 WSS-HS3

Logic and Reasoning Cognitive and General Knowledge IV-A1 Makes comparisons among objects.

14 WSS-HS3

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-E1 Shows understanding of some comparative words.

14 WSS-HS3

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-E2 Participates in measuring activities.

14 WSS-HS3

Science Knowledge and Skills Cognitive and General Knowledge VIII-B1 Uses senses to observe and explore classroom materials and natural phenomena.

14 WSS-HS3

Science Knowledge and Skills Cognitive and General Knowledge VIII-A1 Begins to use simple tools and equipment for investigation.

14 WSS-HS4

Logic and Reasoning Cognitive and General Knowledge IV-A1 Makes comparisons among objects.

14 WSS-HS4

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-E1 Orders, compares, and describes objects according to a single attribute

14 WSS-HS4

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-E2 Participates in measuring activities.

14 WSS-HS4

Science Knowledge and Skills Cognitive and General Knowledge VIII-A1 Uses simple tools and equipment for investigation.

14 WSS-HS4

Science Knowledge and Skills Cognitive and General Knowledge VIII-B1 Asks questions and uses senses to observe and explore materials and natural phenomena.

14 WSS-K Mathematical Thinking Cognitive and General Knowledge III-E1 Orders, compares, and describes objects by size, length, capacity, and weight.

14 WSS-K Mathematical Thinking Cognitive and General Knowledge III-E2 Explores common instruments for measuring during work or play.

14 WSS-K Scientific Thinking Cognitive and General Knowledge IV-A2 Uses simple tools and equipment to extend the senses and gather data.

14 WSS-K Scientific Thinking Cognitive and General Knowledge IV-B1 Identifies, describes, and compares properties of objects.

15 COR Science and Mathematics Cognitive and General Knowledge DD Identifying sequence, change, causality.

15 GOLD Science and Technology Cognitive and General Knowledge 24 Uses scientific inquiry skills.




Match/Objectiv


15 WSS-HS3


15 WSS-HS4


15 WSS-K Scientific Thinking Cognitive and General Knowledge IV-A1 Seeks information through observation, exploration, and descriptive investigations.

16 COR Science and Mathematics Cognitive and General Knowledge EE Identifying materials and properties.

16 GOLD Science and Technology Cognitive and General Knowledge 26 Demonstrates knowledge of the physical properties of objects and materials.



16 WSS-HS3


16 WSS-HS4


16 WSS-K Scientific Thinking Cognitive and General Knowledge IV-B1 Identifies, describes, and compares properties of objects.

16 WSS-K Scientific Thinking Cognitive and General Knowledge IV-D1 Explores and identifies properties of rocks, soil, water, and air.

17 COR Initiative Cognitive and General Knowledge C Initiating play. "pretend play/pretend roles, child modifies play"

17 COR Creative Representation Cognitive and General Knowledge K Pretending.

17 GOLD Cognitive Cognitive and General Knowledge 14b Engages in sociodramatic play. "plans/negotiates complex role play"

17 WSS-P3 The Arts Cognitive and General Knowledge VI-A2 Participates in creative movement, dance, and drama.


17 WSS-HS3

Creative Arts Expression Cognitive and General Knowledge IX-B1 Participates in creative movement, dance, and drama.

17 WSS-HS3

Creative Arts Expression Cognitive and General Knowledge IX-D1 Participates in dramatic play activities

17 WSS-HS4

Creative Arts Expression Cognitive and General Knowledge IX-D1 Engages in dramatic play

17 WSS-HS4



Match/Objectiv


17 WSS-K The Arts Cognitive and General Knowledge VI-A2 Participates in creative movement, dance, and drama.

18 COR Movement and Music Cognitive and General Knowledge O Moving to music.

18 GOLD The Arts Cognitive and General Knowledge 35 Explores dance and movement concepts.



18 WSS-HS3


18 WSS-HS4


18 WSS-K The Arts Cognitive and General Knowledge VI-A2 Participates in creative movement, dance, and drama.

19 COR Movement and Music Cognitive and General Knowledge N Feeling and expressing steady beat.

19 COR Movement and Music Cognitive and General Knowledge P Singing.

19 GOLD The Arts Cognitive and General Knowledge 34 Explores musical concepts and expression.

19 WSS-P3 The Arts Cognitive and General Knowledge VI-A1 Participates in group music experiences.

19 WSS-P4 The Arts Cognitive and General Knowledge VI-A1 Participates in group music experiences.

19 WSS-HS3

Creative Arts Expression Cognitive and General Knowledge IX-A1 Participates in group music experiences.

19 WSS-HS4

Creative Arts Expression Cognitive and General Knowledge IX-A1 Participates in group music experiences.

19 WSS-K The Arts Cognitive and General Knowledge VI-A1 Participates in group music experiences.

20 COR Creative Representation Cognitive and General Knowledge J Drawing and painting pictures.

20 COR Cognitive Cognitive and General Knowledge I Making and building models

20 GOLD Cognitive Cognitive and General Knowledge 14a Thinks Symbolically.

20 GOLD The Arts Cognitive and General Knowledge 33 Explores the visual arts.


Match/Objectiv


20 WSS-HS3

Creative Arts Expression Cognitive and General Knowledge IX-C1 Responds to artistic creations or events.

20 WSS-HS4

Creative Arts Expression Cognitive and General Knowledge IX-C1 Responds to artistic creations or events.

20 WSS-K The Arts Cognitive and General Knowledge VI-B1 Responds to artistic creations or events.

20 WSS-P3 The Arts Cognitive and General Knowledge VI-B1 Responds to artistic creations or events.

20 WSS-P4 The Arts Cognitive and General Knowledge VI-B1 Responds to artistic creations or events.

20 WSS-P3 Language and Literacy Cognitive and General Knowledge II-D1 Represents ideas and stories through pictures, dictation, and play.

20 WSS-P3 The Arts Cognitive and General Knowledge VI-A3 Uses a variety of art materials for tactile experience and exploration.

20 WSS-P4 Language and Literacy Cognitive and General Knowledge II-D1 Represents ideas and stories through pictures, dictation.

20 WSS-P4 The Arts Cognitive and General Knowledge VI-A3 Uses a variety of art materials for tactile experience and exploration.

20 WSS-HS3

Literacy Knowledge and Skills Cognitive and General Knowledge VI-E1 Represents ideas and stories through pictures, dictation, and play.

20 WSS-HS3

Creative Arts Expression Cognitive and General Knowledge IX-C2 Uses a variety of art materials for tactile experience and exploration.

20 WSS-HS3

Logic and Reasoning Cognitive and General Knowledge IV-B1 Represents ideas and stories through pictures, dictation, and play.

20 WSS-HS4

Logic and Reasoning Cognitive and General Knowledge IV-B1 Represents ideas and stories through pictures, dictation and play

20 WSS-HS4

Literacy Knowledge and Skills Cognitive and General Knowledge VI-E2 Represents ideas and stories through pictures, dictation and play

20 WSS-HS4

Creative Arts Expression Cognitive and General Knowledge IX-C2 Uses a variety of art materials for tactile experience and exploration.

20 WSS-K Language and Literacy Cognitive and General Knowledge II-D1 Represents ideas and stories through pictures, dictation, and play.

20 WSS-K The Arts Cognitive and General Knowledge VI-A3 Uses a variety of art materials to explore and express ideas and emotions.

21 COR Science and Mathematics Cognitive and General Knowledge FF Identifying natural and living things.

21 GOLD Science and Technology Cognitive and General Knowledge 27 Demonstrates knowledge of Earth's environment.

21 WSS-P3 Social Studies Cognitive and General Knowledge V-D1 Shows beginning awareness of their environment.


Match/Objectiv


21 WSS-P4 Social Studies Cognitive and General Knowledge V-D2 Shows awareness of the environment.

21 WSS-HS3

Social Studies Knowledge and Skills Cognitive and General Knowledge X-B1 Shows beginning awareness of their environment.

21 WSS-HS4

Social Studies Knowledge and Skills Cognitive and General Knowledge X-B2 Shows awareness of the environment.

21 WSS-K Scientific Thinking Cognitive and General Knowledge IV-D1 Begins to observe and describe simple seasonal and weather changes.

21 WSS-K Scientific Thinking Cognitive and General Knowledge IV-D2 Explores and identifies properties of rocks, soil, water, and air.

22 COR Science and Mathematics Cognitive and General Knowledge FF Identifying natural and living things.

22 GOLD Science and Technology Cognitive and General Knowledge 25 Demonstrates knowledge of the characteristics of living things



22 WSS-HS3


22 WSS-HS4


22 WSS-K Scientific Thinking Cognitive and General Knowledge IV-C1 Observes and describes characteristics, basic needs, and life cycles of living things

23 COR Social Relations Social-Emotional E Relating to adults.

23 GOLD Social Emotional Social-Emotional 2a Forms relationships with adults.

23 GOLD Social Emotional Social-Emotional 1b Follows limits and expectations.

23 WSS-P3 Personal and Social Development Social-Emotional I-B1 Follows simple classroom rules and routines with guidance.

23 WSS-P3 Personal and Social Development Social-Emotional I-E1 Seeks adult help when needed to resolve conflicts.

23 WSS-P3 Personal and Social Development Social-Emotional I-D2 Interacts with familiar adults.

23 WSS-P4 Personal and Social Development Social-Emotional I-B1 Follows simple classroom rules and routines.



Match/Objectiv


23 WSS-P4 Personal and Social Development Social-Emotional I-D2 Interacts easily with familiar adults.

23 WSS-HS3

Social and Emotional Development Social-Emotional II-C1 Follows simple classroom rules and routines with guidance.

23 WSS-HS3

Social and Emotional Development Social-Emotional II-A4 Seeks adult help when needed to resolve conflicts.

23 WSS-HS3

Social and Emotional Development Social-Emotional II-A2 Interacts with familiar adults.

23 WSS-HS4

Social and Emotional Development Social-Emotional II-C1 Follows simple classroom rules and routines.

23 WSS-HS4


23 WSS-HS4

Social and Emotional Development Social-Emotional II-A2 Interacts easily with familiar adults.

23 WSS-K Personal and Social Development Social-Emotional I-B1 Follows classroom rules and routines.

23 WSS-K Personal and Social Development Social-Emotional I-D2 Interacts easily with familiar adults.

23 WSS-K Personal and Social Development Social-Emotional I-E1 Seeks adult help and begins to use simple strategies to resolve conflicts.

24 COR Social Relations Social-Emotional F Relating to other children.

24 GOLD Social Emotional Social-Emotional 2c Interacts with peers.

24 GOLD Social Emotional Social-Emotional 2d Makes friends.

24 GOLD Social Emotional Social-Emotional 3a Balances needs and rights of self and others.

24 WSS-P3 Personal and Social Development Social-Emotional I-D3 Participates in the group life of the class.

24 WSS-P3 Personal and Social Development Social-Emotional I-D1 Interacts with one or more children.

24 WSS-P4 Personal and Social Development Social-Emotional I-D1 Interacts easily with one or more children.

24 WSS-P4 Personal and Social Development Social-Emotional I-D3 Participates in the group life of the class.

24 WSS-HS3

Approaches to Learning Social-Emotional III-C1 Participates in the group life of the class.

24 WSS-HS3

Social and Emotional Development Social-Emotional II-A1 Interacts with one or more children.


Match/Objectiv


24 WSS-HS4

Approaches to Learning Social-Emotional III-C1 Participates in the group life of the class.

24 WSS-HS4

Social and Emotional Development Social-Emotional II-A1 Interacts easily with one or more children.

24 WSS-K Personal and Social Development Social-Emotional I-D3 Participates in the group life of the class.

24 WSS-K Personal and Social Development Social-Emotional I-D1 Interacts easily with one or more children.

25 COR Social Relations Social-Emotional G Resolving interpersonal conflict.

25 GOLD Social Emotional Social-Emotional 3b Solves social problems.



25 WSS-HS3


25 WSS-HS4


25 WSS-K Personal and Social Development Social-Emotional I-E1 Seeks adult help and begins to use simple strategies to resolve conflicts.

26 COR Social Relations Social-Emotional H Understanding and expressing feelings.

26 GOLD Social Emotional Social-Emotional 2b Responds to emotional cues.

26 GOLD Social Emotional Social-Emotional 1a Manages feelings.

26 WSS-P3 Personal and Social Development Social-Emotional I-D4 Shows empathy and caring for others.

26 WSS-P4 Personal and Social Development Social-Emotional I-D4 Shows empathy and caring for others.

26 WSS-HS3

Social and Emotional Development Social-Emotional II-A3 Shows empathy and caring for others.

26 WSS-HS4

Social and Emotional Development Social-Emotional II-A3 Shows empathy and caring for others.

26 WSS-K Personal and Social Development Social-Emotional I-D4 Shows empathy and caring for others.

27 COR Initiative Physical Development and Health D Taking care of personal needs.


Match/Objectiv


27 GOLD Social Emotional Physical Development and Health 1c Takes care of own needs appropriately.

27 GOLD Physical/Special Ed Physical Development and Health 1c1 Eating and drinking.

27 GOLD Physical/Special Ed Physical Development and Health 1c2 Toileting and personal hygiene.

27 GOLD Physical/Special Ed Physical Development and Health 1c3 Dressing.

27 GOLD Physical/Special Ed Physical Development and Health 1c4 Personal Safety.

27 WSS-P3 Physical Development and Health Physical Development and Health VII-C1 Begins to perform self-care tasks independently.

27 WSS-P3 Physical Development and Health Physical Development and Health VII-C2 Follows basic health and safety rules with reminders.

27 WSS-P4 Physical Development and Health Physical Development and Health VII-C1 Performs some self-care tasks independently.

27 WSS-P4 Physical Development and Health Physical Development and Health VII-C2 Follows basic health and safety rules.

27 WSS-HS3

Physical Development and Health Physical Development and Health I-A1 Begins to perform self-care tasks independently.

27 WSS-HS3

Physical Development and Health Physical Development and Health I-A2 Follows basic health and safety rules with reminders.

27 WSS-HS4

Physical Development and Health Physical Development and Health I-A1 Performs self-care tasks independently.

27 WSS-HS4

Physical Development and Health Physical Development and Health I-A2 Follows basic health and safety rules.

27 WSS-K Physical Development and Health Physical Development and Health VII-C1 Performs self-care tasks competently.

27 WSS-K Physical Development and Health Physical Development and Health VII-C2 Shows beginning understanding of and follows health and safety rules.

28 COR Movement and Music Physical Development and Health L Moving in various ways.

28 COR Movement and Music Physical Development and Health M Moving with objects.

28 GOLD Physical Physical Development and Health 4 Demonstrates traveling skills.

28 GOLD Physical Physical Development and Health 5 Demonstrates balancing skills.

28 GOLD Physical Physical Development and Health 6 Demonstrates gross motor manipulative skills.


Match/Objectiv


28 WSS-P3 Physical Development and Health Physical Development and Health VII-A1 Moves with some balance and control.

28 WSS-P3 Physical Development and Health Physical Development and Health VII-A2 Coordinates movements to perform simple tasks.

28 WSS-P4 Physical Development and Health Physical Development and Health VII-A1 Moves with balance and control.

28 WSS-P4 Physical Development and Health Physical Development and Health VII-A2 Coordinates movements to perform simple tasks.

28 WSS-HS3

Physical Development and Health Physical Development and Health I-B1 Moves with some balance and control

28 WSS-HS3

Physical Development and Health Physical Development and Health I-A2 Coordinates movements to perform simple tasks.

28 WSS-HS4

Physical Development and Health Physical Development and Health I-B1 Moves with balance and control.

28 WSS-HS4

Physical Development and Health Physical Development and Health I-B2 Coordinates movements to perform simple tasks.

28 WSS-K Physical Development and Health Physical Development and Health VII-A1 Moves with balance and control.

28 WSS-K Physical Development and Health Physical Development and Health VII-A2 Coordinates movements to perform tasks.

29 COR Movement and Music Physical Development and Health L Moving in various ways.

29 COR Movement and Music Physical Development and Health M Moving with objects.

29 GOLD Physical Physical Development and Health 7a Uses fingers and hands.

29 GOLD Physical Physical Development and Health 7b Uses writing and drawing tools.

29 WSS-P3 Physical Development and Health Physical Development and Health VII-B1 Uses strength and control to perform simple tasks.

29 WSS-P3 Physical Development and Health Physical Development and Health VII-B2 Uses eye-hand coordination to perform simple tasks.

29 WSS-P3 Physical Development and Health Physical Development and Health VII-B3 Explores the use of various drawing and art tools.

29 WSS-P4 Physical Development and Health Physical Development and Health VII-B1 Uses strength and control to perform simple tasks.

29 WSS-P4 Physical Development and Health Physical Development and Health VII-B2 Uses eye-hand coordination to perform tasks.

29 WSS-P4 Physical Development and Health Physical Development and Health VII-B3 Shows beginning control of writing, drawing, and art tools.


Match/Objectiv


29 WSS-HS3

Physical Development and Health Physical Development and Health I-C1 Uses strength and control to perform simple tasks.

29 WSS-HS3

Physical Development and Health Physical Development and Health I-C2 Uses eye-hand coordination to perform simple tasks.

29 WSS-HS3

Physical Development and Health Physical Development and Health I-C3 Explores the use of various drawing and art tools.

29 WSS-HS4

Physical Development and Health Physical Development and Health I-C1 Uses strength and control to perform simple tasks.

29 WSS-HS4

Physical Development and Health Physical Development and Health I-C2 Uses eye-hand coordination to perform tasks.

29 WSS-HS4

Physical Development and Health Physical Development and Health I-C3 Shows beginning control of writing, drawing and art tools.

29 WSS-K Physical Development and Health Physical Development and Health VII-B1 Uses strength and control to accomplish tasks.

29 WSS-K Physical Development and Health Physical Development and Health VII-B2 Uses eye-hand coordination to perform tasks effectively.

29 WSS-K Physical Development and Health Physical Development and Health VII-B3 Uses writing and drawing tools with some control.

30 GOLD Cognitive Approaches to Learning 11a Attends and engages

30 GOLD Cognitive Approaches to Learning 11b Persists

30 GOLD Cognitive Approaches to Learning 11c Solves problems

30 GOLD Cognitive Approaches to Learning 11d Shows curiosity and motivation

30 GOLD Cognitive Approaches to Learning 11e Shows flexibility and inventiveness in thinking

30 COR Initiative Approaches to Learning A Making choices and plans

30 COR Initiative Approaches to Learning B Solving problems with materials

30 COR Initiative Approaches to Learning C Initiating play

30 COR Initiative Approaches to Learning D Taking care of personal needs

30 WSS-P3 Personal and Social Development Approaches to Learning I-C1 Shows eagerness and curiosity as a learner.

30 WSS-P3 Personal and Social Development Approaches to Learning I-C2 Attends briefly, and seeks help when encountering a problem.


Match/Objectiv


30 WSS-P3 Personal and Social Development Approaches to Learning I-C3 Approaches play with purpose and inventiveness.

30 WSS-P3 Personal and Social Development Approaches to Learning I-D3 Participates in the group life of the class.

30 WSS-P4 Personal and Social Development Approaches to Learning I-C1 Shows eagerness and curiosity as a learner.

30 WSS-P4 Personal and Social Development Approaches to Learning I-C2 Attends to tasks and seeks help when encountering a problem.

30 WSS-P4 Personal and Social Development Approaches to Learning I-C3 Approaches tasks with flexibility and inventiveness.

30 WSS-P4 Personal and Social Development Approaches to Learning I-D3 Participates in the group life of the class.

30 WSS-HS3

Approaches to Learning Approaches to Learning III-A1 Shows eagerness and curiosity as a learner.

30 WSS-HS3

Approaches to Learning Approaches to Learning III-A2 Approaches play with purpose and inventiveness.

30 WSS-HS3

Approaches to Learning Approaches to Learning III-B1 Attends briefly, and seeks help when encountering a problem.

30 WSS-HS3

Approaches to Learning Approaches to Learning III-C1 Participates in the group life of the class.

30 WSS-HS4

Approaches to Learning Approaches to Learning III-A1 Shows eagerness and curiosity as a learner.

30 WSS-HS4

Approaches to Learning Approaches to Learning III-A2 Approaches tasks with flexibility and inventiveness.

30 WSS-HS4

Approaches to Learning Approaches to Learning III-B1 Attends to tasks and seeks help when encountering a problem.

30 WSS-HS4

Approaches to Learning Approaches to Learning III-C1 Participates in the group life of the class.

30 WSS-K Personal and Social Development Approaches to Learning I-C1 Shows eagerness and curiosity as a learner.

30 WSS-K Personal and Social Development Approaches to Learning I-C2 Sustains attention to a task, persisting even after encountering difficulty

30 WSS-K Personal and Social Development Approaches to Learning I-C3 Approaches tasks with flexibility and inventiveness.

30 WSS-K Personal and Social Development Approaches to Learning I-D3 Participates in the group life of the class.


Appendix Table A-3. Item Non-Matches

Number Tool Developer’s Domain Common Metric Domain Item ID Item1 GOLD Cognitive Cognitive and General Knowledge 12a Recognizes and recalls

2 GOLD Cognitive Cognitive and General Knowledge 12b Makes connections

3 GOLD Social-Emotional Social-Emotional 29 Demonstrates knowledge about self

4 GOLD Social-Emotional Social-Emotional 30 Shows basic understanding of people and how they live

5 GOLD Social-Emotional Social-Emotional 31 Explores change related to familiar people or places

6 GOLD Cognitive Cognitive and General Knowledge 32 Demonstrates simple geographic knowledge

7 WSS-HS3

Social and Emotional Development Social-Emotional II-B1 Demonstrates self-confidence.

8 WSS-HS3

Social and Emotional Development Social-Emotional II-B2 Shows some self-direction.

9 WSS-HS3

Social and Emotional Development Social-Emotional II-C2 Begins to use classroom materials carefully.

10 WSS-HS3

Social and Emotional Development Social-Emotional II-C3 Manages transitions.

11 WSS-HS3

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-B1 Shows interest in solving mathematical problems.

12 WSS-HS3

Social Studies Knowledge and Skills Cognitive and General Knowledge X-A1 Begins to understand own physical characteristics and those of others.

13 WSS-HS3

Social Studies Knowledge and Skills Cognitive and General Knowledge X-A2 Begins to understand family structures and roles.

14 WSS-HS3

Social Studies Knowledge and Skills Cognitive and General Knowledge X-A3 Describes some jobs people do.

15 WSS-HS3

Social Studies Knowledge and Skills Cognitive and General Knowledge X-A4 Shows awareness of group rules.

16 WSS-HS4

Social and Emotional Development Social-Emotional II-B1 Demonstrates self-confidence.

17 WSS-HS4

Social and Emotional Development Social-Emotional II-B2 Shows some self-direction.

18 WSS-HS4

Social and Emotional Development Social-Emotional II-C2 Uses classroom materials carefully.


Number Tool Developer’s Domain Common Metric Domain Item ID Item19 WSS-

HS4Social and Emotional Development Social-Emotional II-C3 Manages transitions.

20 WSS-HS4

Logic and Reasoning Cognitive and General Knowledge IV-A2 Begins to be aware of technology and how it affects life.

21 WSS-HS4

Mathematics Knowledge and Skills Cognitive and General Knowledge VII-B1 Begins to use simple strategies to solve mathematical problems.

22 WSS-HS4

Social Studies Knowledge and Skills Cognitive and General Knowledge X-A1 Identifies similarities and differences in personal and family characteristics

23 WSS-HS4

Social Studies Knowledge and Skills Cognitive and General Knowledge X-A2 Begins to understand family needs, roles, and relationships.

24 WSS-HS4

Social Studies Knowledge and Skills Cognitive and General Knowledge X-A3 Describes some people's jobs and what is required for them to perform them.

25 WSS-HS4

Social Studies Knowledge and Skills Cognitive and General Knowledge X-A4 Demonstrates awareness of rules.

26 WSS-HS4

Social Studies Knowledge and Skills Cognitive and General Knowledge X-A5 Shows beginning awareness of what it means to be a leader.

27 WSS-HS4

Social Studies Knowledge and Skills Cognitive and General Knowledge X-B1 Describes the location of things in the environment

28 WSS-K Personal and Social Development Social-Emotional I-A1 Demonstrates self-confidence.

29 WSS-K Personal and Social Development Social-Emotional I-A2 Shows initiative and self-direction.

30 WSS-K Personal and Social Development Social-Emotional I-B2 Uses classroom materials purposefully and respectfully

31 WSS-K Personal and Social Development Social-Emotional I-B3 Manages transitions and adapts to changes in routine

32 WSS-K Mathematical Thinking Cognitive and General Knowledge III-A1 Begins to use and explain strategies to solve mathematical problems.

33 WSS-K Mathematical Thinking Cognitive and General Knowledge III-A2 Uses words and representations to describe mathematical ideas

34 WSS-K Mathematical Thinking Cognitive and General Knowledge III-E3 Estimates and measures using nonstandard and standard units

35 WSS-K Mathematical Thinking Cognitive and General Knowledge III-E4 Shows awareness of time concepts

36 WSS-K Mathematical Thinking Cognitive and General Knowledge III-F1 Begins to collect data and make records using lists or graphs

37 WSS-K Scientific Thinking Cognitive and General Knowledge IV-A3 Forms explanations and communicates scientific information

38 WSS-K Social Studies Cognitive and General Knowledge V-A1 Identifies similarities and differences in people's characteristics, habits, and living patterns.


Number Tool Developer’s Domain Common Metric Domain Item ID Item39 WSS-K Social Studies Cognitive and General Knowledge V-A2 Demonstrates beginning awareness of state and country

40 WSS-K Social Studies Cognitive and General Knowledge V-A3 Shows some awareness of time and how the past influences people's lives.

41 WSS-K Social Studies Cognitive and General Knowledge V-B1 Begins to understand how people rely on others for goods and services

42 WSS-K Social Studies Cognitive and General Knowledge V-B2 Describes some people's jobs and what is required to perform them.

43 WSS-K Social Studies Cognitive and General Knowledge V-B3 Begins to be aware of technology and how it affects life.

44 WSS-K Social Studies Cognitive and General Knowledge V-C1 Demonstrates awareness of the reasons for rules.

45 WSS-K Social Studies Cognitive and General Knowledge V-C2 Shows beginning understanding of what it means to be a leader.

46 WSS-K Social Studies Cognitive and General Knowledge V-D1 Expresses geographic thinking

47 WSS-K Social Studies Cognitive and General Knowledge V-D2 Shows beginning awareness of the relationship between people and where they live.

48 WSS-P3 Personal and Social Development Social-Emotional I-A1 Demonstrates self-confidence.

49 WSS-P3 Personal and Social Development Social-Emotional I-A2 Shows some self-direction.

50 WSS-P3 Personal and Social Development Social-Emotional I-B2 Begins to use classroom materials carefully.

51 WSS-P3 Personal and Social Development Social-Emotional I-B3 Manages transitions.

52 WSS-P3 Mathematical Thinking Cognitive and General Knowledge III-A1 Shows interest in solving mathematical problems.

53 WSS-P3 Social Studies Cognitive and General Knowledge V-A1 Begins to understand own physical characteristics and those of others.

54 WSS-P3 Social Studies Cognitive and General Knowledge V-B1 Begins to understand family structures and roles.

55 WSS-P3 Social Studies Cognitive and General Knowledge V-B2 Describes some jobs people do.

56 WSS-P3 Social Studies Cognitive and General Knowledge V-C1 Shows awareness of group rules.

57 WSS-P4 Personal and Social Development Social-Emotional I-A1 Demonstrates self-confidence.

58 WSS-P4 Personal and Social Development Social-Emotional I-A2 Shows some self-direction.


Number Tool Developer’s Domain Common Metric Domain Item ID Item59 WSS-P4 Personal and Social Development Social-Emotional I-B2 Uses classroom materials carefully.

60 WSS-P4 Personal and Social Development I-B3 Manages transitions.

61 WSS-P4 Mathematical Thinking Cognitive and General Knowledge III-A1 Begins to use simple strategies to solve mathematical problems.

62 WSS-P4 Social Studies Cognitive and General Knowledge V-A1 Identifies similarities and differences in personal and family characteristics.

63 WSS-P4 Social Studies Cognitive and General Knowledge V-B1 Begins to understand family needs roles and relationships.

64 WSS-P4 Social Studies Cognitive and General Knowledge V-B2 Describes some people's jobs and what is required to perform them.

65 WSS-P4 Social Studies Cognitive and General Knowledge V-B3 Begins to be aware of technology and how it affects life.

66 WSS-P4 Social Studies Cognitive and General Knowledge V-C1 Demonstrates awareness of rules.

67 WSS-P4 Social Studies Cognitive and General Knowledge V-C2 Shows awareness of what it means to be a leader.

68 WSS-P4 Social Studies Cognitive and General Knowledge V-D1 Describes the location of things in the environment.

69 WSS-P4 Social Studies Cognitive and General Knowledge V-D2 Shows awareness of the environment.


APPENDIX B

Massachusetts EEC Common Metric Progress Memorandum 59Table B1. Differences in Work Sampling System Domain Scores by Age and Quartile, Means and Sample Sizes19

Three-Year Olds Four-Year Olds Five-Year Olds Age Differences

Quartile Differences

Age X QuartileAssessment and

Domains Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

WSS-Personal and Social Development 36.24 44.1 53.9 63.9 35.

1 44.3 53.7 64.2 36.1 44.5 54 63.9ns Q1 < Q2 <

Q3 < Q4 nsN 105 156 84 61 74 142 130 122 104 202 139 164

WSS Language and Literacy 36.3 45.1 52.4 62.8 36.

8 45.2 52.7 64 37.7 44.9 52.7 63.7

3 < 4 < 5 Q1 < Q2 < Q3 < Q4 ns

N 125 114 114 64 54 111 80 133 54 111 62 83

WSS-Mathematical Thinking 35.3 45.3 51.2 63.5 36.

6 45.3 50.4 65.4 38.9 45.3 51.3 62.1

ns Q1 < Q2 < Q3 < Q4 p<.05

N 63 122 69 63 24 94 62 117 54 74 48 25

WSS-Scientific Thinking 35.3 45.4 52.2 65.4 34.2 45.4 52.1 65.4 36.1 45.4 52.

1 65.4ns Q1 < Q2 <

Q3 < Q4 p<.05N 51 188 32 52 51 142 33 130 79 142 33 130

WSS-Social Studies 29.6 40.2 52.5 64.1 32.9 41.2 51.6 64.7 29.7 42.2 60.

0 62.33 < 4 < 5 Q1 < Q2 <

Q3 < Q4 p<.05N 14 36 59 80 14 19 54 113 32 43 23 6

WSS-The Arts 26.6 37.1 44.2 61.2 28.6 37.2 44.2 61.3 28.3 37.2 44.

7 61.1ns Q1 < Q2 <

Q3 < Q4 nsN 17 13 182 99 17 13 182 144 11 15 162 96

WSS-Physical Development and Health 31.3 40.5 50.8 62.6 32.

7 40.5 53 62.6 32.5 40.5 51.5 62.6

ns Q1 < Q2 < Q3 < Q4 p<.05

N 10 123 122 55 13 87 100 108 14 109 102 87

WSS-Total Score 37.7 44.9 52.6 63.2 36.6 45.2 52.9 63.6 37.5 45.1 52.

6 63.3ns Q1 < Q2 <

Q3 < Q4 nsN 136 137 110 70 84 147 127 178 133 192 167 177

WSSHS Physical Development and Health 35.1 44.3 51.3 63.1 39.

5 44.3 50.9 63.43 < 4 Q1 < Q2 <

Q3 < Q4 p<.05N 128 255 109 115 30 169 129 213

WSSHS Social and Emotional Development

36.6 44.4 51.5 62.2 38.4

44.4 52.3 63.2 3 < 4 Q1 < Q2 < Q3 < Q4

ns

19 For the WSS-Head Start and COR, there were no data for five-year olds.

Massachusetts EEC Common Metric Progress Memorandum 60N 196 58 126 115 65 27 124 208

WSSHS Approaches to Learning 34.6 43.6 52.6 62.5 35.

4 43.6 52.6 62.5ns Q1 < Q2 <

Q3 < Q4 nsN 155 76 50 123 48 37 53 219

WSSHS Logic and Reasoning 34.0 42.5 51.1 64.5 37.

7 45.4 51.1 63.83 < 4 Q1 < Q2 <

Q3 < Q4 p<.05N 148 109 272 76 64 65 217 191

WSSHS Language Development 33.5 46.1 54.7 60.4 37.

5 45.8 54.7 61.73 < 4 Q1 < Q2 <

Q3 < Q4 p<.05N 116 136 81 63 40 66 52 152

WSSHS Literacy Knowledge and Skills 35.9 45.5 54.6 62.7 38.

2 47.2 53.7 63.43 < 4 Q1 < Q2 <

Q3 < Q4 p<.05N 196 165 30 94 58 94 49 199

WSSHS Mathematics Knowledge and Skills 36.6 46.4 51.7 62.4 39.

4 46.4 52.1 64.33 < 4 Q1 < Q2 <

Q3 < Q4 p<.05N 195 173 137 101 87 71 144 186

WSSHS Science Knowledge and Skills 28.4 38.1 47.8 64.8 28.

4 38.1 47.8 65.0ns Q1 < Q2 <

Q3 < Q4 nsN 57 76 382 93 4 36 315 186

WSSHS Creative Arts Expression 37.7 46.3 51.5 62.7 38.

5 46.3 52.1 63.23 < 4 Q1 < Q2 <

Q3 < Q4 nsN 206 198 83 120 69 160 97 213

WSSHS Social Studies Knowledge and Skills 38.3 47.2 50.5 62.7 39.

7 47.8 51.2 63.43 < 4 Q1 < Q2 <

Q3 < Q4 nsN 218 166 133 84 68 120 152 198

WSSHS Total Score 37.3 46.4 51.3 63.8 40.1 46.7 51.8 63.9

3 < 4 Q1 < Q2 < Q3 < Q4 p<.05

N 219 168 134 87 68 120 153 201

COR Initiative 35.2 44.8 54.1 60.5 39.3 46.6 54.6 62.9

3 < 4 Q1 < Q2 < Q3 < Q4 ns

N 44 50 13 10 2 44 36 59

COR Social Relations 35.5 46.7 53.6 62.5 36.5 46.6 54.9 62.8

ns Q1 < Q2 < Q3 < Q4 ns

N 47 46 13 12 7 43 40 53COR Creative Representation 35.7 48.8 53.5 61.9 39.

5 48.6 54.2 62.83 < 4 Q1 < Q2 <

Q3 < Q4 nsN 64 17 24 13 4 32 61 51

Massachusetts EEC Common Metric Progress Memorandum 61COR Movement and Music 37.5 47.6 54.4 61.3 42.

3 48.5 54.6 61.73 < 4 Q1 < Q2 <

Q3 < Q4 p<.05N 69 14 7 14 13 26 28 59

COR Language and Literacy 36.2 46.4 54.1 60.4 37.

4 47.0 54.3 62.7ns Q1 < Q2 <

Q3 < Q4 nsN 53 30 11 6 4 18 48 55

COR Mathematics and Science 35.5 45.8 54.4 59.4 38.

8 47.8 54.8 61.8 3 < 4 Q1 < Q2 < Q3 < Q4 ns

N 53 32 17 6 8 31 37 64

COR Total Score 35.8 46.3 54.7 60.4 40.0 48.4 54.5 62.4

3 < 4 Q1 < Q2 < Q3 < Q4 p<.05

N 73 45 25 14 9 38 56 68

Massachusetts EEC Common Metric Progress Memorandum 62Table B2. GOLD Differences in Assessment Domain Scores by Age and Quartile, Means and Sample Sizes.

One-Year Olds Two-Year Olds Three-Year Olds Four-Year Olds Five-Year OldsAge

DifferencesQuartile

DifferencesAge X

QuartileAssessment and Domains Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

Social Emotional 34.3 45.6 53.6 59.2 36.

9 45.8 52.7 62.2 37.1 46.2 53.3 61.4 38.1 46.6 53.8 62.5 38.0 46.9 54.1 62.9

1<2<3<4<5 Q1 < Q2 < Q3 < Q4 p<.05

N 604 46 7 4 618 350 102 33 1,411 1,402 839 391 666 1,301 1,75

0 2,096 106 240 517 1,105

Physical 32.7 46.1 54.3 60.9 36.

9 46.9 54.4 61.1 36.7 47.2 54.3 60.8 38.0 47.6 54.6 61.9 38.5 48.0 54.8 62.6

1<2,3<4,5 Q1 < Q2 < Q3 < Q4 p<.05

N 744 61 13 5 547 422 75 46 1,383 1,570 587 361 619 1,380 1,54

5 1,983 74 239 381 1,176

Language 32.5 45.0 54.0 59.8 36.

7 45.9 53.4 60.4 36.4 46.5 53.9 60.6 37.7 46.8 54.3 61.9 38.2 47.0 54.4 62.2

1<2<3<4<5 Q1 < Q2 < Q3 < Q4 p<.05

N 616 23 6 5 602 345 84 31 1,314 1,411 871 404 598 1,204 1,77

7 2,177 90 280 526 968

Cognitive 33.4 45.1 52.5 58.5 36.

9 45.6 53.0 59.6 37.1 46.3 53.0 61.2 38.3 46.8 53.6 62.7 38.3 47.0 53.9 63.0

1<2<3<4<5 Q1 < Q2 < Q3 < Q4 p<.05

N 542 20 7 2 679 317 62 20 1,321 1,512 743 346 509 1,232 1,73

0 2,100 76 245 540 931

Literacy 36.2 46.6 53.6 57.3 37.

7 44.5 51.6 62.3 38.0 45.1 52.3 60.8 38.8 45.5 52.5 63.0 39.5 45.8 52.7 64.4

1<2<3<4<5 Q1 < Q2 < Q3 < Q4 p<.05

N 68 5 3 3 405 122 20 4 1,384 1,094 433 208 617 1,250 1,40

7 1,684 44 275 451 890

Mathematics 33.5 1 45.1 61.4 36.5 44.8 53.1 60.

4 37.4 45.6 53.1 60.7 38.2 46.2 53.7 62.9 38.8 46.4 53.8 63.91<2<3<4<5 Q1 < Q2 < Q3

< Q4 p<.05 N 215 1 10 2 625 219 26 5 1,301 1,438 553 234 525 1,227 1,503 1,911 65 240 379 886

1 Excluded due to small cell count.

Massachusetts EEC Common Metric Progress Memorandum 63Figure B1. WSS Preschool interaction effects of age by quartile on assessment scores by domain


Figure B2. WSS Head Start interaction effects of age by quartile on assessment scores by domain


Figure B3. COR interaction effects of age by quartile on assessment scores by domain


Figure B4. GOLD interaction effects of age by quartile on assessment scores by domain

Web viewA tenth domain, English language acquisition, is included to assess language skills for ......

Documents

Transcript of Web viewA tenth domain, English language acquisition, is included to assess language skills for ......