Development, Reliability and Validity Testing of Toddler Children … · 2015. 8. 24. ·...

Draft

Development, Reliability and Validity Testing of Toddler

NutriSTEP®: A Nutrition Risk Screening Questionnaire for Children 18-35 Months of Age

Journal: Applied Physiology, Nutrition, and Metabolism

Manuscript ID: apnm-2015-0048.R1

Manuscript Type: Article

Date Submitted by the Author: 06-Apr-2015

Complete List of Authors: Randall Simpson, Janis; University of Guelph

Gumbley, Jillian; University of Guelph, Family Relations and Applied Nutrition Whyte, Kylie; University of Guelph, Family Relations and Applied Nutrition Lac, Jane; University of Guelph, Family Relations and Applied Nutrition Morra, Crystal; Noojmowin Teg Health Centre, Rysdale, Lee; University of Guelph, Family Relations and Applied Nutrition Turfryer, Mary; York Region Health Services, McGibbon, Kim; Thunder Bay District Health Unit, Beyers, Joanne; Sudbury & District Health Unit, Keller, Heather; University of Waterloo, Kinesiology

Keyword: nutrition < nutrition, feeding behavior < behavior, behavioural nutrition < nutrition, growth, pediatrics < medicine

https://mc06.manuscriptcentral.com/apnm-pubs

Applied Physiology, Nutrition, and Metabolism

Draft

1

Development, Reliability and Validity Testing of Toddler NutriSTEP®: A Nutrition Risk

Screening Questionnaire for Children 18-35 Months of Age

Authors:

Randall Simpson, Janis, University of Guelph, Guelph, Ontario, N1G 2W1, Canada

[email protected]

Gumbley, Jillian, University of Guelph, Guelph, Ontario, N1G 2W1, Canada

[email protected]

Whyte, Kylie, University of Guelph, Guelph, Ontario, N1G 2W1, Canada [email protected]

Lac, Jane, University of Guelph, Guelph, Ontario, N1G 2W1, Canada [email protected]

Morra, Crystal, Noojmowin Teg Health Centre, Little Current, Ontario, P0P 1K0, Canada

[email protected]

Rysdale, Lee, University of Guelph, Guelph, Ontario, N1G 2W1, Canada [email protected]

Turfryer, Mary, York Region Public Health, Newmarket, Ontario, L3Y 6Z1, Canada

[email protected]

McGibbon, Kim, Thunder Bay District Health Unit, Ontario, P7B 6E7, Canada

[email protected]

Beyers, Joanne, Sudbury & District Health Unit, Sudbury, Ontario, P3E 6H3 Canada

[email protected]

Keller, Heather, Schlegel-University of Waterloo Research Institute for Aging, University of

Waterloo, Waterloo, Ontario, N2L 3G1 Canada [email protected]

Corresponding Author:

Janis Randall Simpson, Department of Family Relations and Applied Nutrition, University of Guelph,

50 Stone Road East, Guelph, Ontario, Canada N1G 2W1. Phone: 519 924-4120, ext 53843. FAX:

519 766-0691. E-mail: [email protected].

Page 1 of 43



Draft

2

Funding:

Canadian Institutes of Health Research, University of Guelph, University of Guelph-Humber

Conflict of Interest:

Janis Randall Simpson, Lee Rysdale and Heather Keller receive royalties from the sale of the version

of Toddler NutriSTEP® that has scoring options for use by implementers of the NutriSTEP® screening

program at: www.Flintbox.com.

A version without scoring options is available free of charge for parents at: www.nutritionscreen.ca.

Page 2 of 43



Draft

3

Abstract

Background /Objectives

Nutrition is vital for optimal growth and development of young children. Nutrition risk screening can

facilitate early intervention when followed by nutritional assessment and treatment. NutriSTEP® is a

valid and reliable nutrition risk screening questionnaire for preschoolers (3-5 years). A need was

identified for a similar questionnaire for toddlers (18-35 months). The purpose was to develop a

reliable and valid Toddler NutriSTEP®.

Subjects/Methods

Toddler NutriSTEP® was developed in four phases. Content and face validity were determined with a

literature review, parent focus groups (n=6, 48 participants) and experts (n=13) (Phase A). A draft

questionnaire was refined with key intercept interviews of 107 parents/caregivers (Phase B). Test-

retest reliability (Phase C), based on intra-class correlations (ICC), Kappa (ĸ) statistics, and Wilcoxon

tests was assessed with 133 parents/caregivers. Criterion validity (Phase D) was assessed using

Receiver Operating Characteristic (ROC) curves by comparing scores on the Toddler NutriSTEP® to a

comprehensive nutritional assessment of 200 toddlers with a registered dietitian (RD).

Results

The Toddler NutriSTEP® was reliable between two administrations (ICC=0.951, F=20.53, p< 0.001);

most questions had moderate (ĸ≥0.6) or excellent (ĸ≥0.8) agreement. Scores on the RD nutrition risk

rating and the Toddler NutriSTEP® were correlated (r=0.67, p<0.000). The area under the ROC curve

for moderate and high RD risk ratings were 84.6% and 82.7%, respectively. Cut-points of ≥21

(sensitivity 86%; specificity 61%) (moderate risk) and ≥26 (sensitivity 95%; specificity 63%) (high

risk) were determined.

Conclusions

Page 3 of 43



Draft

4

The Toddler NutriSTEP® questionnaire is both reliable and valid for screening for nutritional risk in

toddlers.

Running Title:

Development of Toddler NutriSTEP®

Key Words:

Nutrition risk screening, toddlers, validity, reliability, NutriSTEP®

Page 4 of 43



Draft

5

Introduction

Nutrition is vital for optimal growth and development. Unhealthy feeding/eating environments (e.g.,

inappropriate foods served or lack of division of responsibility) put children at risk for poor nutrition

(Satter 1995; Savage et al. 2007). Young children with poor nutrition are at risk for growth,

behavioural and developmental problems, including overweight/obesity (Thompson et al. 2014).

Delayed, inadequate or inappropriate nutrition intervention and management has both short- and long-

term health consequences for growing children, including iron deficiency (Christofides et al. 2005;

Paoletti et al. 2014), and lay the foundation for chronic diseases such as diabetes, heart disease, obesity,

and osteoporosis (Halfon et al. 2012).

Nutrition screening identifies those at nutritional risk who may be asymptomatic, using

characteristics of known nutrition problems for the purpose of assessment and treatment ("ethical

screening") (American Dietetic Association 1994; Kondrup et al. 2003; Rush 1997). Because

eating habits and patterns are established at an early age, it is important to address the

foundations of healthy eating and a healthy lifestyle during the toddler and preschool periods.

Thus, screening has this secondary purpose of increasing awareness of potential nutritional

problems for parents and care providers. Screening to facilitate early action, where intervention

may be less intensive and costly than assessment, is crucial in effective primary health care

models (Kondrup et al. 2003).

NutriSTEP® (Nutrition Screening Tool for Every Preschooler) is a valid and reliable 17-item,

community-based, parent-completed questionnaire for identifying nutritional risk in preschoolers (3-5

years) (Randall Simpson et al. 2008). NutriSTEP® can be used to address nutrition concerns with

parents; further, the use of NutriSTEP® in public health settings can provide data for monitoring and

surveillance. NutriSTEP® has been implemented in Canadian public health jurisdictions including

Page 5 of 43



Draft

6

Ontario as an accountability indicator (Ontario 2014) and New Brunswick (New Brunswick Public

Health 2012), and is used in research and primary care (Persaud et al. 2013; Watson-Jarvis et al. 2011).

Since the release of the preschool NutriSTEP®, other pediatric nutrition risk screening tools have been

developed, but are for use in acute/hospital settings (Gerasimidis et al. 2010; Hulst et al. 2010;

McCarthy et al. 2012; Secker and Jeejeebhoy 2007). A gap continues to exist for identifying nutrition

risk in community-living toddlers. A recently-developed Australian questionnaire focuses on toddlers’

food group intake (Bell et al. 2014); however, it does not include other diverse nutrition risk factors,

such as physical growth and development, physical activity and sedentary behaviour, and family

factors related to nutrition. Other toddler questionnaires address only caregiver feeding behaviours and

practices (Chaidez et al. 2011; Hurley et al. 2013) and not other nutrition risk factors.

During the implementation of the preschool version of NutriSTEP®, exploring the validation of

NutriSTEP® for children of 18 months of age was suggested (Watson-Jarvis et. 2011), as this age group

has unique nutritional issues and risks that are different from preschoolers. Additionally, a senior

public health official in Ontario (S. Basrur (personal communication, 2006)) suggested that a nutrition

risk questionnaire for this age group was needed to complement universal screening for 18 month old

children in Ontario that began in 2009 (Ontario Ministry of Children and Youth Services 2011).

The preschool NutriSTEP® has four risk factor areas (attributes): food and fluid intake; physical

growth; physical activity and sedentary behaviour; and, factors affecting food intake (Randall Simpson

et al. 2008). All of these attributes are also relevant to nutrition risk in toddlers and thus the preschool

NutriSTEP® could be used as an initial starting point for the toddler version. While some questions, or

modifications of questions, on the preschool NutriSTEP® are appropriate for toddlers, there are

developmental issues for toddlers that are not addressed in the preschool version. Based on the

literature and on expert and parent/caregiver opinions, additional concerns, unique to toddlers, such as

Page 6 of 43



Draft

7

developmental milestones related to food and fluid consumption, and feeding relationships were

considered for inclusion.

The overall purpose of this article is to describe the development, reliability and validity testing of

Toddler NutriSTEP® for children 18-35 months of age. The ultimate goals of this screening tool are

three-fold: to identify toddlers who require a nutrition assessment to diagnose and treat their impaired

state; to provide referrals for needed services; and, to track population-level nutrition risk in toddlers.

Subjects and Methods

Design

Toddler NutriSTEP® was developed in four phases (Figure 1) (Phase A: Face and Content Validity;

Phase B: Refinement; Phase C: Reliability; and, Phase D: Validation), using a combined

clinimetric (reliance on intuition and experience to create clinically useful instruments),

psychometric (reliance on the generation of many question items to address attributes of areas of

interest) and criterion (validation compared to a 'gold standard’) method (Keszei et al. 2010; Streiner

and Norman 2008). The project was modeled on the development of the original preschool NutriSTEP®

(Randall Simpson et al. 2008) and adapted from Keller et al. (2000; 2001) and Jones (2004a).

Subjects

For all phases, convenience samples of parents/caregivers of toddlers from ethnically, geographically,

and socio-demographically diverse backgrounds in rural and urban areas throughout Southern and

Northern Ontario, Canada were recruited by research assistants at Ontario Early Years Centres and at

Peer Nutrition (programs through Toronto Public Health). To ensure representativeness, and thus

applicability, to potentially vulnerable groups, populations with possible low and high nutrition risk

(including European, Middle Eastern, Asian, African and Aboriginal Canadians) were specifically

recruited. Inclusion criteria for participants were that they were parents/primary caregivers of toddlers

and able to read and write English at a grade six level. Residence in Canada for at least five years was

Page 7 of 43



Draft

8

also an inclusion criteria for participants in the first three phases. As demographic and socioeconomic

diversity was desired for validation, this inclusion criterion was not used in Phase D and sites in

multiethnic Toronto were used for recruitment. Exclusion criteria included hospitalized and

institutionalized children. Ethics clearance was provided by the University of Guelph Research Ethics

Board. Participants provided informed signed consent and completed a demographic form used

previously (Randall Simpson et al. 2008; Rysdale et al. 2011). Incentives included grocery vouchers

(Phases A, B, C: $20 CDN; Phase D: $40 CDN) and parent education materials. Focus group

participants in Phase A received a healthy snack.

Methods

Phase A: Face and Content Validity

The objective of this phase was to determine and confirm what constitutes nutritional risk in toddlers

and to draft the Toddler NutriSTEP®.

A literature review was conducted to assess the current expert understanding of what constitutes toddler

nutrition risk and to determine if there were any existing nutrition risk questionnaires for this age group

(Streiner and Norman 2008). Parents/caregivers were then recruited for focus groups to confirm their

concerns and risk behaviours for this age group (Kreuger and Casey 2009; Randall Simpson et al.

2008). Six one-hour focus groups were conducted by a trained moderator and two trained note takers;

a pilot-tested standardized script about the suitability of the NutriSTEP® preschool questions for

toddlers was used, considering possible deletions or additions for the toddler age group. Debriefing

amongst researchers after each focus group identified if changes were required to the script to focus the

discussion and to develop further probes. A single transcriptionist transferred audio to text; transcripts

were checked for errors and were reviewed by core members of the research team (JRS, JG, LR, HK,

JB). Key findings from these focus groups and from the literature review became the basis for the draft

Toddler NutriSTEP®.

Page 8 of 43



Draft

9

The content of the draft Toddler NutriSTEP® was reviewed by 13 (an appropriate number (Streiner and

Norman 2008)) registered dietitians (RDs), with expertise in pediatrics and with a variety of work

experience from across Canada. These experts not only confirmed the content of the questions but also

verified language to promote clarity (Keszei et al. 2010). An online survey (pre-tested with nutrition

graduate students) was used to rank draft questions based on their relative importance to further

confirm content and identify items less relevant for potential exclusion. Further, a group

teleconference with the same RDs, using a discussion guide, was used to address each proposed

Toddler NutriSTEP® question, clarifying concerns or language, to reach consensus on final questions.

Phase B. Refinement

The objective of this phase was to refine and finalize the draft Toddler NutriSTEP®.

Key intercept interviews were conducted by six trained research assistants to ensure that wording of

question stems and response options was understandable, user friendly, and culturally-appropriate to

diverse parents. The goal was to recruit approximately 100 participants (Streiner and Norman 2008).

These parents first completed the draft Toddler NutriSTEP® and then were asked a series of open-

ended questions using a standardized interview guide about each question stem to confirm the expected

cognitive processing (Randall Simpson et al. 2008). Participants' understanding, thoughts, and

opinions of the question stems, possible examples, and question response options were recorded.

Using qualitative content analysis (Harris et al. 2009), the feedback was reviewed and consideration

was given to any potential changes to item stems, examples and response categories.1

Phase C: Test-retest reliability

The objective of this phase was to determine the test-retest reliability of Toddler NutriSTEP®.

1 The question stems are given in Supplementary Material (Table S1).

Page 9 of 43



Draft

10

A sample size of approximately140 participants was desired and considered more than adequate based

on a desired Kappa (ĸ ) value (0.8) and anticipated prevalence of risk (30-40%) (Jones 2004b; Randall

Simpson et al. 2008; Streiner and Norman 2008). At recruitment, participants completed the draft

Toddler NutriSTEP®; approximately 2-4 weeks later (Jones 2004b), participants completed the draft

questionnaire for the second time.

Phase D: Validation

The primary objective of this phase was to determine criterion validity of Toddler NutriSTEP® relative

to a nutrition risk assessment by an RD, considered to be the 'gold standard' (American Dietetic

Association 1994; Jones 2004c; Streiner and Norman 2008).

Sample size for criterion validity was estimated at between 110-200 participants based on: 1) estimates

for a Receiver Operating Characteristic (ROC) curve analysis (~110) with area under the curve (AUC)

of 85% (Streiner and Cairney 2007); and, 2) estimates of specificity of 75%, risk of 26%, error of 10%

and 95% confidence interval (~215) (Jones 2004c). Our goal was set at 200 participants.

The validation phase involved two visits with the parents/caregivers and toddlers by one RD with

expertise in pediatrics who conducted a comprehensive nutritional assessment of the toddlers. At the

first visit, parents/caregivers completed the draft Toddler NutriSTEP® which was then placed into a

sealed envelope so that the RD was blinded to responses. Toddlers’ weights were measured (to 50 g)

in triplicate using a calibrated scale, with a removable weighing tray for younger toddlers (Tanita

Digital Baby Scale (1584), Arlington Heights, IL, USA). Height (to 0.1 cm) for toddlers ≥24 months

of age was measured using a portable stadiometer (Road Rod (214), SECA, Chino, CA, USA); length

(to 0.1 cm) was measured using a portable measuring mat for toddlers <24 months of age (Starters

Pediatric Measure Mat, Slater & Frith, Norwich, UK). Weight for length (<2 years), and BMI for age

(≥2 years) was assessed using the 2010 World Health Organization growth standards adapted for

Page 10 of 43



Draft

11

Canada (Dietitians of Canada 2014). Finally, parents/caregivers were given instructions to complete a

three-day food record.

Within one month, the second visit was completed. The food record was reviewed and consumption

amounts and patterns were compared with Canada's Food Guide (Health Canada 2012). The RD

completed a comprehensive content-valid assessment interview using a risk-rating guide2, based on the

literature review (Phase A), and adapted from the preschool version (Randall Simpson et al. 2008). The

RD risk rating was a 10-point scale used in previous research (low risk (1-4); moderate risk (5-7); high

risk (8-10)) (Keller et al. 2001; Keller et al. 2005; Randall Simpson et al. 2008). Construct validity was

also determined through comparison of the researchers' theoretical expectations of the questionnaire

across demographic groups with the performance of the tool itself (Jones 2004c).

Data analysis

All quantitative data were entered in duplicate to check for accuracy. Data analysis was conducted

with SPSS (version 20, IBM Inc., Chicago, IL, USA) or SAS (version 9.3, SAS Systems Inc., Cary,

NC, USA). Demographic information was analyzed using descriptive statistics. Test-retest reliability

was assessed via intra-class correlations (ICC), Wilcoxon signed rank tests, Pearson's correlation, and a

paired sample t-test (Jones 2004b; Streiner and Norman 2008). Kappa statistics, ranging from 0-1, with

higher values indicating greater reliability, were also calculated for individual questions (dichotomized

into 'risk' and 'no risk') to determine if they were significantly different across the two occasions (Jones

2004b). To assess validity, scores (0-68) on the Toddler NutriSTEP® were compared to the RD risk

rating using ROC curves that were created using moderate (≥5) and high risk (≥8) cut-points on the 10-

ten point RD rating scale (Streiner and Cairney 2007; Streiner and Norman 2008). A high AUC

indicates that the measured variable is consistent with scoring of the criterion (Streiner and Norman

2008). Appropriate risk cut-points for Toddler NutriSTEP® were identified by comparing trade-offs of

2 Supplementary Material Table S2

Page 11 of 43



Draft

12

sensitivity (Se) and specificity (Sp) for various scores using the ROC curves output, considering as well

the original preschool NutriSTEP® scoring. To assess construct validity, t-tests and analysis of

variance were used to compare the average response on the questionnaire across varying risk groups to

see if hypothesized trends existed (Jones 2004c; Streiner and Norman 2008).

Results

Subjects

As shown in Table 1, most parents/caregivers were in their early 30s, female, married, with post-

secondary education, and with household incomes >$60,000 per year. Many parents/caregivers and

children in Phases A, B and D did not speak English as their first language. Many parents/caregivers,

particularly in Phases A and D, were not born in Canada, whereas most of the toddlers in all phases

were born in Canada. There was no attrition of participants in Phases A and B.

Phases A and B

Literature regarding NutriSTEP® nutrition risk issues in toddlers was limited, with most North

American data collected from just two studies: Continuing Survey of Food Intakes by Individuals

(CSFII) (Kranz et al. 2004) and Feeding Infants and Toddlers Study (Devaney et al. 2003). These

surveys provided information on issues specific to toddlers that were not already considered in the

preschool NutriSTEP®. A number of nutrition risk issues, specifically for toddlers, were identified:

changes in milk (including breastmilk and formula) consumption (Hawkins and Law, 2005; Siega-Riz

et al. 2010; Skinner et al. 2004); introduction of inappropriate table foods (Briefel et al. 2004; Butte et

al. 2010; Fox et al. 2004); excess intake of fruit juices and sweetened beverages (Fox et al. 2004; Fox et

al. 2006; Nicklas and Hayes 2008; Siega-Riz et al. 2010); picky eating (Carruth et al. 2004);

development of inappropriate parent-toddler feeding relationships (Satter 1990); and, prolonged bottle

and/or “sippy-cup” use (Bonuck et al. 2004). As well, no existing nutrition risk screening tool for

community-living toddlers was identified.

Page 12 of 43



Draft

13

Forty-eight parents/caregivers participated in the six focus groups. Participants confirmed the

importance of almost all preschool NutriSTEP® questions, with some modifications. They also

identified non-nutrition issues unique to toddlers that can impact a child’s weight such as too much

time spent in car seats/strollers and poor sleep routines. Based on the literature review and the focus

group feedback, a 21-item Toddler NutriSTEP® was drafted incorporating the developmental issues of

toddler nutrition. After expert consultation, this was reduced to 19 items for the next phase (Phase B).

Key intercept interviews were conducted with 107 parents/caregivers of toddlers. Major suggestions

were the inclusion of more examples and clarification of some terms (e.g., fast food). Consideration of

these comments and suggestions resulted in a final 17-item draft Toddler NutriSTEP® used in Phases C

and D. The final draft included the following: four new toddler-specific questions (e.g., drinking from

a baby bottle; child feeding his/her self); six questions modified from the original preschool

NutriSTEP® for toddlers (e.g., including breast milk as an example for the question on consumption of

milk products); and, seven original preschool NutriSTEP® questions (e.g., questions on: frequency of

consumption of food groups from Canada's Food Guide; food security; frequency of consumption of

fast food; amount of "screen time").

Phase C

Of the 158 parents/caregivers recruited, 141 completed the draft Toddler NutriSTEP® on two visits, on

average 17±1 days apart; 133 had complete data that were used for this analysis. Total mean scores

(±SD) on the Toddler NutriSTEP® (15±7 for both; range 3-34 of a possible 68) were found to be highly

reliable (ICC=0.951, F=20.53, p<0.001). The Pearson's correlation on the total Toddler NutriSTEP®

score between administrations was significant (r=0.91, p<0.001) and the paired sample t-test was not

significantly different over time (t-statistic=0.882, p>0.05).

All dichotomized responses to individual questions had significant ĸ statistics (p<0.001), with most

showing at least fair reliability (ĸ>0.4) (Table 2). Further, most questions had non-significant

Page 13 of 43



Draft

14

Wilcoxon tests, indicating reliability on all response options, with only three questions having

significant differences: grain product consumption, milk product consumption, and the child deciding

how much to eat.

Phase D

Of the 218 families recruited for the validation phase, 200 completed both visits. The mean (±SD)

score on Toddler NutriSTEP® was 19.8±8.7 with a range of 3-46 (of a possible 68). Items that placed

toddlers in this phase at risk were: low consumption of grains, fruits and vegetables and meat and

alternatives; finding food expensive; drinking from a bottle with a nipple; not being hungry at meals;

and, weight concerns.

The ROC curve for Toddler NutriSTEP® scores compared to the RD high risk scores (score ≥8) is

shown in Figure 2 with an AUC of 82.7%. At the moderate risk cut-point for the RD rating (score ≥5),

the AUC was 84.6% (ROC curve not shown). Data for Se and Sp for the moderate and high risk RD

cut points (scores ≥5 and ≥8, respectively), as well as the prevalence of toddlers at various Toddler

NutriSTEP® scores, are shown in Table 3. Cut-points were determined for three risk classification

categories as follows: low risk (score ≤20); moderate risk (score 21-25); and, high risk (score ≥26).

Based on these data, a cut point of ≥21 has Se of 86% and Sp of 61% compared to the RD moderate

risk rating and Se of 71% and Sp of 80% compared to the RD high risk rating. A cut point of ≥26 has

Se of 95% and Sp of 43% compared to the RD moderate risk rating and Se of 83% and Sp of 63%

compared to the RD high risk rating.

The mean (±SD) RD risk score was 4.8±2.5. The Spearman's rho between the scores on Toddler

NutriSTEP® and the RD risk rating scores was 0.67 (p<0.000). Further, the mean scores for the RD-

rated risk categories for Toddler NutriSTEP®, based on cut-points in Table 3, were significantly

different (F=349.3, p<0.000), indicating that NutriSTEP® risk classifications were reflective of the

RD risk ratings.

Page 14 of 43



Draft

15

Results of the tests for construct validity are shown in Table 4. As hypothesized, NutriSTEP® scores

were significantly lower (lower risk) for parents born in Canada, with higher household income, and

higher levels of education.

Discussion

The major outcome of this research is the successful development of a valid and reliable nutrition risk

questionnaire for toddlers. The validation results (i.e., AUC 83% and 85% for moderate and high risk,

respectively) are slightly better than for the original NutriSTEP® (i.e., AUC 82% and 74% for moderate

and high risk rating, respectively) (Randall Simpson et al. 2008). Furthermore, the Se and Sp at the

final chosen cut-points for the toddler version are comparable to the preschooler version. The

sensitivity of Toddler NutriSTEP® (>86%) is higher than for malnutrition screening tools (59-70%)

designed for hospitalized children (Gerasimidis et al. 2010; McCarthy et al. 2012).

The validation was conducted on a diverse sample, providing a wide range of nutrition risk scores.

There was a considerable prevalence of high nutritional risk (27%) for children in Phase D. This was

considerably higher than for Phase C (7%) and in comparison to the preschool NutriSTEP® validation

sample (14%) (Randall Simpson et al. 2008), and other NutriSTEP® research based on convenience

samples (7%) (Rysdale et al. 2011; Watson-Jarvis et al. 2011). Differences in the prevalence of

problematic issues by item for Phases C and D are also different and are likely attributed to successful

recruitment of a very diverse sample from urban Toronto for Phase D. For example, significantly more

mothers in Phase C versus Phase D were born in Canada, had English as their first language, had a

higher level of education and higher household income (all p<0.002). As expected, based on

previously- documented associations between socioeconomic status and nutrition (Bradley and Corwyn

2002; Hendricks et al. 2006), NutriSTEP® scores were lower for those toddlers from families with

higher income and higher level of education. Children of immigrant parents have been reported to be

at nutritional risk because of many issues, including food insecurity and consumption of foods high in

Page 15 of 43



Draft

16

energy and fat (Flaskerud and Kim 1999; Satia-Abouta et al. 2002). Our findings reflect the

relationships between education, income, immigration status and nutritional risk of children seen in the

literature, and therefore demonstrate construct validity of the Toddler NutriSTEP®. There was a higher

prevalence of toddlers classified as overweight and/or obese (43%) in the validation sample relative to

the preschool validation sample (21%) which was less diverse (Randall Simpson et al. 2008), and to

Canadian preschool children in the CCHS (26%) (Shields 2006). Demographic differences were also

reflected in the greater number of questions with a high prevalence of risk for the validation sample

(Phase D) versus the reliability sample (Phase C).

Common areas of concern included low frequency of consumption of grain products (47-64%,

for Phases C and D, respectively), also reported in a national sample of Canadian toddlers (Kirkpatrick

and Tarasuk 2008). More than 30% drank from a baby bottle with a nipple, even though the transition

from bottle feeding to an open cup is recommended to be complete by 18 months of age (Bonuck et al.

2004; Health Canada 2014). Over 33% watched TV or played video games more than two hours per

day whereas recent Canadian recommendations are for less than one hour per day (Canadian Society

for Exercise Physiology 2012). These results outline areas of modifiable nutrition risk and provide

evidence to support public health and primary health care interventions directed at parents/caregivers of

toddlers.

The Toddler NutriSTEP® is considered reliable based on analyses of total score and on individual

questions. These findings are consistent with the original preschool NutriSTEP® (Randall Simpson et

al. 2008). Reliability is slightly higher (ICC=0.95) than the Australian food group questionnaire

(ICC=0.90) (Bell et al. 2014) and higher than other toddler risk tools for hospitalized children

(Gerasimidis et al. 2010; McCarthy et al. 2012). Based on the ĸ values for the dichotomized individual

questions, and on the Wilcoxon tests, the question on frequency of grain consumption was less reliable

Page 16 of 43



Draft

17

than the others. This question may require future refinement to promote reliability, if a true variability

in intake was not the reason for this difference.

Two other questions (frequency of milk consumption; child decides how much to eat) also

demonstrated some variability in repeat administration as analyzed with Wilcoxon tests. However,

both questions had significant and adequate ĸ statistics (0.70 and 0.53, respectively) indicating that,

although the absolute score on the question from time one to time two differed, the risk classification

for that question did not. Therefore, these two questions are considered to have adequate reliability, as

they are still able to reliably classify a child into a particular risk-rating category across administrations.

Strengths of this work include the rigorous development, reliability and criterion validation testing of

Toddler NutriSTEP® and inclusion of a diverse sample, including Aboriginal Peoples. Other recent

dietary/feeding practice questionnaires for toddlers have been developed in more specialized groups

such as Latino mothers (n=94) in the United States (Chaidez et al. 2011), low-income African

American mothers (n=297) (Hurley et al. 2013), and relatively advantaged samples in Australia

(n=111) (Bell et al. 2014). Toddler NutriSTEP® is novel in that it is comprehensive and includes

attributes (e.g., physical growth and development, physical activity and sedentary behaviour, and

family factors related to nutrition) in addition to dietary intake that differentiates it from the Australian

food-group-based dietary questionnaire, that was validated relative to a food frequency questionnaire

(Bell et al. 2014). Further, the Toddler NutriSTEP® assesses comprehensive nutrition risk compared to

other toddler questionnaires that focus only on caregiver feeding practices and behaviour. For

example, Chaidez et al. (2011) developed a 34-item questionnaire with construct validation of

indulgent, authoritative and environmental feeding practices of caregivers; and, Hurley et al. (2013)

developed a 27-item questionnaire with construct validation of five caregiver feeding behaviours

(responsive, forceful/pressuring, restrictive, indulgent, uninvolved). A further strength is that a single

RD conducted all 200 validation assessments thereby minimizing variability in the criterion

Page 17 of 43



Draft

18

assessment. The sample size for all phases met or exceeded recommendations and was sufficient to

answer the research questions. This study is not, however, without limitations. As convenience and

purposive samples were recruited, it is not surprising that demographics were not consistent with the

Canadian population; the reliability sample had higher levels of education and the validation sample

had a higher representation of vulnerable groups compared to the Canadian population (Statistics

Canada 2011). As such, prevalence estimates of nutrition risk from these studies need confirmation in

a population-level representative sample of Canadian toddlers; however, the validity of the tool is not

affected by this sampling bias. Key intercept interviews, although completed by highly-trained

interviewers, were likely completed by engaged parents who may be atypical. Nevertheless, quality

insights into comprehension of the questions were provided from this group and the reliability and

validation samples did not identify any further challenges with the questionnaire itself.

Conclusions

Toddler NutriSTEP® is a rigorously-developed, valid and reliable nutrition risk screening tool for use

by parents of children aged 18-35 months. This tool fills an important research and practice gap as well

as offers potential for population health surveillance and monitoring of nutrition risk. Well-baby visits

become less frequent after the second year of life. Toddlers continue to transition fully to table foods

and to develop self-feeding practices. Without on-going monitoring, potential unhealthy behaviours can

develop with increased nutritional and subsequent health problems. The prevalence of nutrition risk in

this group suggests a need for systematic screening programs in community settings. Toddler

NutriSTEP® is an easy-to-use tool for this purpose. Further research will include predictive validation

and evaluation of the effectiveness of Toddler NutriSTEP®.

Page 18 of 43



Draft

19

Acknowledgements

The contributions of the following are gratefully acknowledged by the authors. Michelle Edwards

(Statistician, University of Guelph), Elizabeth Shaver Heeney (City of Hamilton Public Health

Services), Heather St. Bernard John (Toronto Public Health), University of Guelph students (Kimberly

Booker, Lauren Baker, Alison Weber), University of Guelph-Humber student (Kimberlee Rizun-

Glynn), Northern Ontario Dietetic Internship Program students (Stephanie Hill, Riley Fulkerson, Jenna

Campbell, Tammy Vachon), and the many Ontario Early Years Centres across Ontario that participated

in this research.

Conflict of Interest

Janis Randall Simpson, Lee Rysdale and Heather Keller receive royalties from the sale of the version

of Toddler NutriSTEP® that has scoring options for use by implementers of the NutriSTEP® screening

program at: www.Flintbox.com. However, an online version without scoring options is available free

of charge for parents at: www.nutritionscreen.ca.

Page 19 of 43



Draft

20

References

ADA (American Dietetic Association). 1994. Identifying patients at risk: ADA’s definitions for

nutrition screening and nutrition assessment. Council on Practice (COP) Quality Management

Committee. J. Am. Diet. Assoc. 94: 838-839. PMID: 8046173.

Bell, L.K., Golley, R.K., and Magarey, A.M. 2013. A short food-group-based dietary questionnaire

is reliable and valid for assessing toddlers' dietary risk in relatively advantaged samples. Br. J.

Nutr. J2: 1-11. doi: 10.1038/ejcn.2013.23.

Bonuck, K.A., Huang, V., and Fletcher, J. 2010. Inappropriate bottle use: An early risk for

overweight? Literature review and pilot data for a bottle-weaning trial. Matern. Child Nutr. 6(1):

38-52. doi: 10.1111/j.1740-8709.2009.00186.x.

Bradley, R.H., and Corwyn, R.F. 2002. Socioeconomic status and child development. Ann. Rev.

Psychol. 53: 371-399. PMID:11752490.

Briefel, R.R., Reidy, K., Karwe, V., Jankowski, L., and, Hendricks, K. 2004. Toddlers’ transition to

table foods: Impact on nutrient intakes and food patterns. J. Am. Diet. Assoc. 104(1): S38-S44.

PMID:14702016.

Butte, N.F., Fox, M.K., Briefel, R.R., Siega-Riz, A.M., Dwyer, J.T., Deming, D.M., et al. 2010.

Nutrient intakes of US infants, toddlers, and preschoolers meet or exceed dietary reference intakes.

J. Am. Diet. Assoc. 110(12): S27-S37. doi: 10.1016/j.jada.2010.09.004.

Page 20 of 43



Draft

21

Canada. Statistics Canada. 2011. Employment and Social Development Canada. Ottawa: Statistics

Canada; Table 202-0605 - Median after-tax income, by economic family type, constant dollars,

annual (dollars). Indicators of Well-Being in Canada [online]. Available from

http://www40.statcan.ca/z01/cs0002-eng.htm [accessed 3 February 2015].

Canadian Society for Exercise Physiology. 2012. Canadian Physical Activity Guidelines -

Background Information [online]. Available from http://www.csep.ca/english/view.asp?x=587

[accessed 2 February 2015].

Carruth, B.R., Ziegler, P.J., Gordon, A., and Barr, S.I. 2004. Prevalence of picky eaters among

infants and toddlers and their caregivers' decisions about offering a new food. J. Am. Diet. Assoc.

104(1 Suppl 1): s57-64. PMID:14702019.

Chaidez, V., and Kaiser, L.L. 2011. Validation of an instrument to assess toddler feeding practices

of Latino mothers. Appetite. 57:229-236. PMID:21600943.

Christofides, A., Schauer, C., and Zlotkin, S.H. 2005. Iron deficiency and anemia prevalence and

associated etiologic risk factors in First Nations and Inuit communities in northern Ontario and

Nunavut. Can. J. Public Health 96: 304-307. PMID:16625803.

Devaney, B., Kalb, L., Briefel, R., Zavitshy-Novak, T., Clusen, N., and Ziegler, P. 2004. Feeding

infants and toddlers study: overview of the study design. J. Am. Diet. Assoc. 104, Suppl 1: s8-13.

PMID:14702012.

Page 21 of 43



Draft

22

Dietitians of Canada. 2014. WHO Growth Charts Adapted for Canada. Summary of Changes -

March 2014. [online]. Available from

http://www.dietitians.ca/Downloads/Public/WHO-Growth-Charts-Summary-of-Change-March-

2014.aspx [accessed 31 March 2015].

Edington, J. Problems of nutritional assessment in the community. 1999. Proc. Nutr. Soc. 58; 47-

51. PMID:10343339.

Flaskerud, J.H., and Kim, S. 1999. Health problems of Asian and Latino immigrants. Nurs. Clin.

North Am. 34(2): 359-380. PMID:10318729.

Fox, M.K., Pac, S., Devaney, B., and Jankowski, L. 2004. Feeding infants and toddlers study: What

foods are infants and toddlers eating? J. Am. Diet. Assoc. 104(1): S22-S30. PMID:14702014.

Fox, M.K., Reidy, K., Novak, T., and Ziegler, P. 2006. Sources of energy and nutrients in the diets

of infants and toddlers. J. Am. Diet. Assoc. 106(1 Suppl 1): S28-S42. PMID:16376628.

Gerasimidis, K., Keane, O., Macleod, I., Flynn, D.M., and Wright, C.M. 2010. A four stage

evaluation of the Paediatric Yorkhill Malnutrition Score in a tertiary paediatric and district

general hospital. Br. J. Nutr. 104: 751-756. doi: 10.1017/S0007114510001121.

Halfon, N., Verhoef, P.A., and Kuo, A.A. 2012. Childhood antecedents to adult cardiovascular

disease. Pediatr. Rev. 33(2): 51-60. doi: 10.1542/pir.33-2-51.

Page 22 of 43



Draft

23

Harris, J.E., Gleason, P.M., Sheean, P.M., Boushey, C., Beto, J.A., and Bruemmer, B. 2009. An

introduction to qualitative research for food and nutrition professionals. J. Am. Diet. Assoc. 109:

80-89. doi: 10.1016/j.jada.2008.10.018.

Hawkins, S.S., and Law, C. 2005. A review of risk factors for overweight in preschool children: A

policy prospective. Int. J. Pediatr. Obes. 1: 195-209. PMID:17907326.

Health Canada. 2012. Ottawa: Canada's Food Guide [online]. Available from http://hc-sc.gc.ca/fn-

an/food-guide-aliment/index-eng.php [accessed 2 February 2015].

Health Canada. 2014. Food and Nutrition; p.540-541. Nutrition for Healthy Term Infants:

Recommendations from Birth to Six Months. A joint statement of Health Canada, Canadian

Paediatric Society, Dietitians of Canada, and Breastfeeding Committee for Canada; 2014 May 27

[online]. Available from http://www.hc-sc.gc.ca/fn-an/nutrition/infant-nourisson/recom/index-

eng.php [accessed 3 February 2015].

Hendricks, K., Briefel, R., Novak, T., and Ziegler, P. 2006. Maternal and child characteristics

associated with infant and toddler feeding practices. J. Am. Diet. Assoc. 106(1 Suppl 1): S135-148.

PMID:16376637.

Hulst, J.M., Zwart, H., Hop, W.C., and Joosten, K.F.M. 2010. Dutch national survey to test

the STRONGkids nutritional risk screening tool in hospitalized children. Clin. Nutr. 29:

106-111. doi: 10.1016/j.clnu.2009.07.006.

Page 23 of 43



Draft

24

Hurley, K.M., Pepper, M.R., Candelaria, M., Wang, Y., Caulfield, L.E., Latta, L., et al. 2013.

Systematic development and validation of a theory-based questionnaire to assess toddler feeding. J.

Nutr. 143:2014-2049. PMID:24068792.

Jones, J.M. 2004a. Development of a nutritional screening or assessment tool using a multivariate

technique. Nutrition 20: 298-306. PMID:14990272.

Jones, J.M. 2004b. Reliability of nutritional screening and assessment tools. Nutrition 20: 307-311.

PMID:14990273.

Jones, J.M. 2004c. Validity of nutritional screening and assessment tools. Nutrition 20: 312-317.

PMID:14990274.

Keller, H.H., Hedley, M.R., and Wong Brownley, S. 2000. The development of seniors in the

community: Risk evaluation for eating and nutrition (SCREEN). Can. J. Diet. Pract. Res. 61(2): 67-

72. PMID: 11551350.

Keller, H.H., McKenzie, J.D., and Goy, R.E. 2001. Construct validation and test-retest reliability of

the seniors in the community: Risk evaluation for eating and nutrition questionnaire. J. Gerontol.

56A(9): M552-M558. PMID:11524447.

Keller, H.H., Goy, R., and Kane, S.L. 2005. Validity and reliability of SCREEN II (Seniors in the

community: Risk evaluation for eating and nutrition, version II). Eur. J. Clin. Nutr. 59: 1149-1157.

PMID:16015256.

Page 24 of 43



Draft

25

Kirkpatrick, S.I., and Tarasuk, V. 2008. Food insecurity is associated with nutrient inadequacies

among Canadin Adults and Adolescents. (Online supporting material) J. Nutr. 138: 604-612.

PMID:18287374.

Kondrup, J., Allison, S.P., Elia, M., Vellas, B., and Plauth, M. 2003. ESPEN Guidelines for

Nutrition Screening 2002. Clin. Nutr. 22(4): 415-421. PMID:12880610.

Kranz, S., Siega-Riz, A.M., and Herring, A.H. 2004. Changes in diet quality of American

preschoolers between 1977 and 1998. Am. J. Public Health 94: 1525-1530. PMID:15333309.

Kreuger, R.A., and Casey, M.A. 2009. Focus groups. A practical guide for applied

research. 4th edition. Sage Publications, Thousand Oaks, CA.

McCarthy, H., Dixon, M., Crabtree, I., Eaton-Evans, M.J., and McNulty, H. 2012. The

development and evaluation of the Screening Tool for the Assessment of Malnutrition in

Paediatrics (STAMP) for use by healthcare staff. J. Hum. Nutr. Diet. 25: 311-318. doi:

10.1111/j.1365-277X.2012.01234.x.

New Brunswick Public Health. 2012. New Brunswick Public Health Nutrition Framework for

Action 2012-2016. 2012 [online]. Available from

http://www2.gnb.ca/content/dam/gnb/Departments/h-

s/pdf/en/Publications/PublicHealthNutritionFrameworkforAction.pdf [accessed 2 February 2015].

Page 25 of 43



Draft

26

Nicklas, T.A., and Hayes, D. 2008. Position of the American Dietetic Association: nutrition

guidance for healthy children ages 2 to 11 years. J. Am. Diet. Assoc. 108(6): 1038-44, 1046-7.

PMID:18564454.

Ontario. 2014. Technical Document: Health Promotion Division 2014. Public Health Funding

and Accountability Agreement Indicators. Toronto: Ministry of Health and Long-Term Care Health

Promotion Division, Toronto, Canada.

Ontario Ministry of Children and Youth Services. 2011. Ontario's Enhanced 18-month Well Baby

Visit [online]. Available from

http://www.children.gov.on.ca/htdocs/English/topics/earlychildhood/health/enhanced_18-

month.aspx [accessed 3 February 2015].

Paoletti, G., Bogen, D.L., and Ritchey, A.K. 2014. Severe iron-deficiency anemia still an issue in

toddlers. Clin. Pediatr. 53(14): 1352-1358. doi: 10.1177/0009922814540990.

Persaud, N., Maguire, J.L., Lebovic, G., Carsley, S., Khovratovich, M., Randall Simpson J.A., et al.

2013. Cholesterol associated with eating behaviors during early childhood. CMAJ 185(11):E531-

536. doi: 10.1503/cmaj.121834.

Randall Simpson, J.A., Keller, H.H., Rysdale, L.A., Beyers, J.E. 2008. Nutrition Screening Tool for

Every Preschooler (NutriSTEP™): validation and test-retest reliability of a parent-administered

questionnaire assessing nutrition risk of preschoolers. Eur. J. Clin. Nutr. 62: 770-780.

PMID:17554250.

Page 26 of 43



Draft

27

Rush, D. 1997. Nutrition screening in old people: its place in the coherent practice of preventive

health care. Ann. Rev. Nutr. 17: 101-125. PMID:9240921.

Rysdale, L., Witchell, E., Keller, H., Randall Simpson, J., Turfryer, M., McGibbon, K., et al. 2011.

Parent perceptions of NutriSTEP and nutrition risk screening in a sample of Ontario preschool

children. Infant, Child and Adolescent Nutrition 3; 197-204. PMID: 21815309.

Satia-Abouta, J., Patterson, R.E., Newhouser, M.L., and Elder, J. 2002. Dietary acculturation:

applications to nutrition research and dietetics. J. Am. Diet. Assoc. 1002(8): 1105-1118.

PMID:12171455.

Satter, E. 1990. The feeding relationship: problems and interventions. J. Pediatr. 117: S181-S189.

PMID:2199651.

Satter E. Feeding dynamics: helping children to eat well. 1995. J. Pediatr. Health Care 9: 178-184.

PMID:7629684.

Savage, J.S., Fisher, J.O., and Birch, L.L. 2007. Parental influence on eating behavior: conception

to adolescence. J. Law Med. Ethics 35: 22-34. PMID:17341215.

Secker, D.J., and Jeejeebhoy, K.N. 2007. Subjective Global Nutritional Assessment for children.

Am. J. Clin. Nutr. 85: 1083-1089. PMID:17413109.

Shields, M. 2006. Overweight and obesity among children and youth. Health Rep. 17(3), 27-42.

Page 27 of 43



Draft

28

Siega-Riz, A.M., Deming, D.M., Reidy, K.C., Fox, M.K., Condon, E., and Briefel, R.R. 2010.

Food consumption patterns of infants and toddlers: Where are we now? J. Am. Diet. Assoc.

110(12): S38-S51. doi: 10.1016/j.jada.2010.09.001.

Skinner, J.D., Ziegler, P., and Ponza, M. 2004. Transitions in infants’ and toddlers’ beverage

patterns. J. Am. Diet. Assoc. 104(1): S45-S50. PMID:14702017.

Streiner, D.L., and Cairney, J. 2007. What's under the ROC? An introduction to receiver operating

characteristic curves. Can. J. Psychiatry 52: 121-128. PMID:17375868.

Streiner, D., and Norman, G.R. 2008. Health measurement scales: A practical guide to their

development and use (4th ed.). Oxford University Press, New York, NY.

Thompson, J., Manore, M., and Sheeshka, J. 2014. Nutrition for preschoolers, age 4-5 years. In

Nutrition: A functional approach. 3rd Canadian ed.. Pearson Education Canada, Toronto, Canada.

pp. 540-541.

Watson-Jarvis, K., Fenton, T.S., McNeil, D., and Campbell, K. 2011. Preschool nutrition risk in

Calgary. Can. J. Diet. Pract. Res. 72: 33101-e3106. PMID:21382227.

Page 28 of 43



Draft

Table 1. Demographic characteristics of parents/caregivers and toddlers in the

development of Toddler NutriSTEP®*.

______________________________________________________________________________

Parent/Caregiver Characteristics Phase A Phase B Phase C Phase D

Focus

Groups

Key-

Intercept

Interviews

Reliability Reliability

n = 48 n = 107 n =133 n =200

mean ± SD mean ± SD mean ± SD mean ± SD

Parent age (years) 34±12 32±7 34±9 32±6

Number of household members 4±1 NA 4±1 4±1

Number of children in household 2±1 2±1 2±1 2±1

n (%) n (%) n (%) n (%)

Gender

Male 6 (13) 6 (6) 11 (8) 15 (8)

Female 42 (87) 101 (94) 122 (92) 182 (91)

Marital Status

Married/partnered 45 (96) 84 (79) 118 (89) 168 (84)

Single/separated/divorced/widowed 2 (4) 22 (21) 15 (11) 30 (15)

Education†

< High school 6 (13) 10 (9) 11 (8) 30 (15)

Graduated high school 3 (6) 12 (11) 5 (4) 16 (8)

Some post-secondary 8 (17) 15 (14) 19 (14) 38 (19)

Page 29 of 43



Draft

Graduated post-secondary 30 (64) 70 65) 98 )74) 116 (58)

Income (CDN) †

<$30,000 15 (34) 0 21 (16) 84 (42)

$30,000 - 59,000 10 (23) 38 (36) 20 (15) 39 (20)

$60,000 - 89,000 19 (43) 21 (20 ) 35 (26) 29 (15)

>$90,000 0 25 (33) 41 (31) 31 (16)

Mother's first language†

English 25 (54) 73 (68) 122 (92) 104 (52)

French 5 (10) 16 (15) 3 (2) 0

Other 17 (35) 18 (17) 8 (6) 96 (48)

Mother born in Canada† 28 (58) 87 (81) 121 ( 91) 100 (50)

Child Characteristics Phase A Phase B Phase C Phase D

mean ± SD mean ± SD mean ± SD mean ± SD

Age (months) 27.2±5.6 25.8±4.8 25.1±5.1 25.4±5.1

Body mass index‡

(for toddlers ≥2 years)

n (%)

Normal 114 (57)

<3rd percentile 0

>85th - 97th %ile 58 (29)

> 97th %ile 28 (14)

Gender n (%) n (%) n (%) n (%)

Male 25 (52) 61 (57) 68 (51) 99 (50)

Page 30 of 43



Draft

Female 23 (48) 46 (43) 64 (48 101 (51)

Age (months)

18 - 23 14 (29) 38 (36) 60 (45) 77 (39)

24-29 12 (25) 42 (39) 61 (38) 67 (34)

30 - 35 22 (46) 27 (25) 26 (20) 56 (28)

First language

English 23 (48) 80 (75) 122 (92) 108 (54)

French 3 (6) 9 (8) 1 (1) 0

Other 15 (31) 9 (8) 9 (7) 92 (46)

Born in Canada 42 (88) 105 (98) 123 (92) 169 (85)

______________________________________________________________________________

* Percentages may not add up to 100% due to rounding errors and/or missing data

† Significantly different (p<0.05) between Phases C & D

‡ BMI based on 2010 WHO growth curves adapted for Canada (Dietitians of Canada 2014)

Page 31 of 43



Draft

Table 2. Proportion of respondents with individual Toddler NutriSTEP® items indicating

risk for reliability (Phase C) and validity (Phase D) Studies and reliability statistics.

______________________________________________________________________________

Phase C Phase C Phase C PhaseD

NutriSTEP© Item Reliability Kappa*

statistic

Wilcoxcon

Signed

Rank

Validity

(n=133) (n=200)

% %

Grains

>3 x per daya 53 0.39 -2.161† 36

≤3 x per dayb 47 64

Milk/Substitute Products

≥3 x per daya 88 0.70 -2.185† 76


Vegetables & Fruit

≥3 x per daya 77 0.68 -0.448 62


Meat & Alternatives

≥2 x per daya 74 0.44 0.00 61

≤1 per dayb 26 39

Fast Food

≤1 x per weeka 71 0.63 -0.622 80

Page 32 of 43



Draft

≥2 x per weekb 29 20

Flavoured Beverages

≤1 x per daya 75 0.67 -0.011 69

≥2 x per dayb 52 31

Food is Expensive

Never to rarelya 73 0.74 -0.698 55

Sometimes to most of the timeb 27 45

Chewing/swallowing difficulty


Sometimes to alwaysb 5 10

Feeding his/her self

Most of the time to alwaysa 90 n/a‡ -0.333 67

Sometimes to neverb 10 33

Drinks from a baby bottle

with a nipple


Sometimes to alwaysb 29 37

Hungry at mealtimes

Sometimes to alwaysa 27 0.49 -0.836 56

Never to rarelyb 73 44

Eating frequency

≥3-≤6 x per daya 89 0.46 -1.043 77

≤2 x per day or >6 x per dayb 11 23

Page 33 of 43



Draft

Child controls amount

consumed

Always/most of the timea 89 0.53 -2.532 69

Sometimes to neverb 11 31

Watches television at meals

Never to sometimesa 86 0.70 -0.870 75

Most of the time to alwaysb 14 25

Sedentary activity during day

≤1 hour per daya 64 36 -1.253 67

≥2 hours per dayb 36 33

Growth adequate§

Yesa 90 10 -1.134 76

Nob 10 24

Weight§

Appropriatea 81 0.82 -1.098 65

Too much/littleb 19 35

_____________________________________________________________________________________________________________________

a Indicates no risk

b Indicates risk

* All kappa statistics were significant at p<0.001. Kappa statistics are: fair (≥0.4); moderate

(≥0.6); excellent (≥0.8)

† Significant Wilcoxcon test at p=0.05

Page 34 of 43



Draft

‡Unable to calculate a kappa statistic because risk/no risk was consistent across administrations

§ Based on parental perception of adequate growth/weight

Page 35 of 43



Draft

Table 3. Cumulative prevalence of participants at/above Toddler NutriSTEP®

score and

comparison of sensitivity (Se) and specificity (Sp) using the moderate and high risk

dietitian cut-points as a comparison for individual Toddler NutriSTEP®

score values.

______________________________________________________________________________

Toddler

NutriSTEP®

Score

Cumulative

Prevalence of

Participants at/above

Toddler NutriSTEP®

Score

Moderate Risk ≥5 High Risk ≥8

Phase D Phase D

Validation Study Validation Study

Reliability

Study*

Validation

Study

Se Sp Se Sp

n=133 n=200

(%) (%) (%) (%) (%) (%)

3 97 99.5 1 100 1 100

4 96.2 98.5 3 100 2 100

5 94 97 6 100 4 100

6 91.7 96.5 7 100 4 100

7 88.7 94.5 11 100 6 100

8 84.2 93 14 100 8 100

Page 36 of 43



Draft

9 78.1 92 16 100 9 100

10 72.9 90 20 100 12 100

11 66.2 84 32 100 19 100

12 57.9 78 42 98 26 100

13 48.1 72.5 50 96 32 100

14 40.6 70.5 54 94 35 97

15 36.1 65 56 97 41 97

16 30.8 59.5 63 84 45 90

17 28.6 54 71 81 52 90

18 24.8 50 73 75 56 87

19 19.5 44.5 77 68 61 80

20 15.8 40 82 65 66 80

21 37 86 61 71 80

22 14.3 36 88 61 72 80

23 12.8 34 90 59 74 80

24 9 31 92 56 76 77

25 6 27 95 51 80 70

26 24 95 43 83 63

27 22 95 40 85 63

28 5.3 20.5 95 32 88 53

29 4.5 15 98 29 91 50

30 3 21.5 98 22 93 40

31 2.3 20.5 99 20 94 37

Page 37 of 43



Draft

32 9 100 18 96 37

33 0 7.5 100 15 97 34

34 0 6.5 100 13 98 30

35 0 6 100 12 98 27

36 0 5 100 10 98 20

37 0 3.5 100 7 98 13

38 0 1.5 100 3 99 6

39 0 0.5 100 2 100 6

43 0 0.5 100 1 100 3

46 0 0.5 100 0 100 0

____________________________________________________________________________

* Values are for the first administration of the questionnaire

Page 38 of 43



Draft

Table 4. Construct validation statistics for Toddler NutriSTEP® .

______________________________________________________________________________

Construct Toddler

NutriSTEP®

score

Overall Model

(mean±SD)

Household Income

<$30,000 (n=78) 23±8

$30,000-$59,999 (n=39 19±8 F=27.68, p<0.001

>$60,000 (n=60) 14±6c

Level of Education

< Graduated high school (n=29) 24±7a

Graduated high school (n=15) 21±6b F=3.96, p=0.009

Graduated post-secondary (n=114) 18±9c

Immigration Status (Mother)

Born in Canada (n=96) 15±6a t=-7.67, p<0.001

Immigrated to Canada (n=95) 24±8b

Immigration Status (Father)

Born in Canada (n=93 15±6a t=-7.00, p<0.001

Immigrated to Canada (n=85) 23±9b

Immigration Status (both parents)

Both parents born in Canada (n=85) 15±6a

One parent born in Canada (n=16) 15±6a F=35.42, p<0.001

Neither parent born in Canada (n=77) 24±8b

_____________________________________________________________________________________________________________________

Page 39 of 43



Draft

a,b,c Values in the same column with different superscripts are significantly different

Page 40 of 43



Draft

1

Figure Captions

Figure 1. Phases in the development of Toddler NutriSTEP®

Figure 2. Receiver operator characteristic (ROC) curves to Toddler NutriSTEP®

score relative to

nutritional risk rating (risk ≥8 on a scale of 1 to 10) by a registered dietitian

Page 41 of 43



Draft

Figure 1. Phases in the Development of Toddler NutriSTEP® 204x164mm (300 x 300 DPI)

Page 42 of 43



Draft

Figure 2. Receiver operator characteristic (ROC) curves to Toddler NutriSTEP® score relative to nutritional risk rating (risk ≥ 8 on a scale of 1 to 10) by a registered dietitian

161x158mm (300 x 300 DPI)

Page 43 of 43



Development, Reliability and Validity Testing of Toddler Children … · 2015. 8. 24. ·...

Documents

Transcript of Development, Reliability and Validity Testing of Toddler Children … · 2015. 8. 24. ·...