Leslie Cunningham-Sabo, PhD, RDN Associate Professor Dept ......Sep 28, 2015 · Retention interval...
Transcript of Leslie Cunningham-Sabo, PhD, RDN Associate Professor Dept ......Sep 28, 2015 · Retention interval...
September 28, 2015 Webinar Leslie Cunningham-Sabo, PhD, RDN
Associate Professor Dept. Food Science & Human Nutrition, Colorado State University
At the end of this webinar attendees will be able to:
Describe at least 3 challenges researchers face when developing valid measures of child dietary intake
Distinguish the definitions and processes to measure face and content validity
Identify the purpose and effective strategies for conducting cognitive interviews with children
Describe at least 1 strategy to assess the sensitivity of an instrument to measure real change
Validation of a group-administered pictorial dietary recall with 9- to 11-year-old children
Instrument: Day in the Life Questionnaire – Colorado (DILQ -CO)
Child’s age ◦ Recall and cognitive ability; 7+ Y.O. generally accepted
Errors include omissions (exclusions) and intrusions; amount eaten
Retention interval and prompts (Baxter, et al., 2015)
◦ Parent/caregiver as proxy – may not observe all meals/snacks (Byers, et
al., 1993)
Usual methods ◦ 24-hr recall, usually 2 week days, 1 weekend day, multiple pass
◦ FFQ and screeners (Hunsberger, et al, 2012)
◦ Method/s selected dependent upon nutrients or dietary groups/patterns of interest
Instrument validity – ability of the instrument to measure what is intended ◦ 4 types ◦ Subsequent webinars will discuss construct and criterion validity
Content – confirm all items reasonably represent larger domain ◦ Clarify concept/construct through literature review ◦ expert (panel) review
Face – determine language, format & procedures are understandable & reasonable to target audience ◦ testing with target audience ◦ When assessing for children can also include adults (parents, teachers)
(Contento, 2015; Sharma & Petosa, 2014)
Purpose ◦ Assess language, format & processes are understood by target audience ◦ One way to assess face validity
Processes ◦ Done individually, not in groups ◦ Administer instrument as intended ◦ Followed by ‘think-aloud’ and probing questions to assess comprehension
and any missing content (e.g., response options)
(Beatty & Willis, 2007; Drennan, 2003; Shafer & Lohse, 2005)
Awareness of developmental stages of cognitive abilities is critical; influenced by age, experience ◦ Age of 7 generally accepted as earliest to interview ◦ Language expands, reading skills acquired, distinguish points of view
Youth continue to develop cognition and communication skills through adolescence and early adulthood
Establish trust; undistracted & unhurried environment
Model type of questions to be asked; probing necessary (de Leeuw, Borgers & Smits, 2004)
Does the instrument detect a true condition? ◦ Often discussed for biological tests in comparison to specificity
◦ Determined by ‘true positives’ divided by sum of true positives and false negatives (TP / TP + FN)
(Sharma & Petosa, 2014)
Rationale – identify feasible method to collect children’s individual self-report of FV intake in a group (classroom) setting
Original DILQ ◦ Development – yesterday’s report of food intake and physical activities
◦ Face validity with focus groups
◦ Performed well/acceptably for
reliability, validity, sensitivity
(Edmunds & Ziebland, 2002)
Similar rationale to original DILQ; assess all food groups
Purpose – ◦ Assess content & face validity of DILQ adapted to Denver, CO 4th-
graders – DILQ-CO
◦ Assess feasibility of administration and analysis of DILQ-CO
◦ Determine DILQ-CO’s sensitivity to novel eating occasion & accuracy of children’s recall of school lunch
Procedures ◦ Child nutrition education research expert review
Content, order of items, language & graphics (US v. UK)
How to include estimation of amounts of foods eaten?
Is forward or reverse recall (end/start after lunch/recess) more comprehensible? (Baxter et al., 2003; Moore et al., 2007)
◦ Elementary teacher review Language, format, feasibility of classroom administration
◦ Cognitive interviews with children of similar age & demographics DILQ item interpretation & comprehension
Format, instructions
Ability to quantify amounts of foods eaten
Wording, phrasing & item order ◦ For example replaced “pudding” with “dessert” & “school dinner, pack lunch” with
“lunch from home, school lunch”
◦ Made embedded instructions more direct (e.g., “check box”)
Added semi-quantitative scale for portions of recalled items ◦ Circle = ate all, √ = ate more than 1/2, x = ate less than 1/2, line through = ate
none
Created 2 versions – either could be administered after lunch/recess ◦ Forward recall order (yesterday to today); reverse recall order (today to yesterday)
Adapted original DILQ administration protocol
10 4th-grade students from school with similar demographic ◦ 8 girls, 2 boys; ½ received forward v reverse format
Applied standardized interview protocol
In general students enjoyed & comprehended DILQ-CO ◦ Including quantification scale
◦ Memory perceived as better with reverse recall order (from lunch/recess today backwards to after lunch/recess yesterday); supported by literature (Baxter, et al. 2003, Baxter, et al., 2009)
Results indicated minor revisions to wording and graphics ◦ E.g., changed front yard gate to house door; evening meal to dinner
2 low-income Denver, CO schools; 1 randomly assigned to receive morning fresh fruit snack (FSS, NFSS)
125 4th graders in 6 classrooms ◦ Power calculation to detect moderate effect size indicated sample >
115
> 77.5% eligible for F/R-priced lunches; 51% male; > 77% Hispanic
DILQ-CO completed after lunch/recess 35 minutes to complete per classroom; few problems or
questions
Procedures ◦ FSS students were offered pre-portioned banana or grapes for morning
snack
◦ We observed and recorded amount wasted
Analysis ◦ Averaged reported instances of fruit snack between schools
◦ Man-Whitney U test
(Penkilo & Hoelscher, 2008)
Procedures ◦ Observation of lunch (recorded item selection/brought from home)
◦ Weighed plate waste
◦ Compared against standard weight samples (school lunches)
Analysis ◦ Food selection/plate waste compared to DILQ-CO for school lunch
◦ Matches = item recorded/not recorded on both data sources
◦ Exclusions = item missing from DILQ-CO
◦ Intrusions = item only recorded on DILQ-CO (inaccurate recall)
◦ Sensitivity = percent matches/(percent matches + percent exclusions)
Students accurately recalled “instances” of morning fruit intake:
FSS 1.18 + 0.93 v NFF 0.02 + 0.13 (mean + SD; P < .001)
DILQ-CO demonstrated high level of validity & sensitivity compared to plate waste (Table 1) ◦ K coefficients of substantial to almost perfect agreement for entrees,
fruit, vegetables, bread, and milk
◦ Only dessert (cookie) had poorer agreement
Accurate recall for specific fruit & vegetable types
Significant correlations between quantified & weighed amounts for 6 of 10 food items (P < .05)
Poorer correlations with vegetables, turkey wrap, milk, bread
No differences by gender or school
Feasible DILQ-CO classroom administration
Students were quite accurate with recall of recent lunch & fruit snack selections ◦ recall of more distal timeframes unknown
Amounts consumed were less accurate & varied by food type ◦ No clear pattern; this warrants further investigation
Challenges with analyses – extensive time to score/code
Inconsistent validation (studies) using original DILQ Moore, et al., 2007 reported comparable k coefficients & percent matches
Lim, et al., 2015, results do not support validity of DILQ
Recommendations for collecting children’s self-report ◦ Reverse recall for immediate 24 hour period
◦ Parent-involved report for non-school meals
Strategies to measure FV change that don’t rely on self-report Plate waste (Smith & Cunningham-Sabo, 2014)
Skin carotenoid assessment of FV intake (Mayne, 2013; Aguilar, 2014)
Article co-authors ◦ Victoria (Wallen) Lee
◦ Garry Auld
◦ Cathy Romaniello
Graduate students ◦ Tessa Komine – cognitive interviewing resources
◦ Alicia Grove – presentation preparation
Aguilar, Sheryl S., et al. "Skin Carotenoids: A Biomarker of Fruit and Vegetable Intake in Children." Journal of the Academy of Nutrition and Dietetics 114.8 (2014): 1174-1180.
Baxter, Suzanne D., et al. "Effectiveness of prompts on fourth-grade children’s dietary recall accuracy depends on retention interval and varies by gender." The Journal of nutrition 145.9 (2015): 2185-2192.
Baxter, Suzanne Domel, et al. "Fourth-grade children's dietary recall accuracy is influenced by retention interval (target period and interview time)." Journal of the American Dietetic Association 109.5 (2009): 846-856.
Baxter, Suzanne Domel, et al. "Reverse versus forward order reporting and the accuracy of fourth-graders’ recalls of school breakfast and school lunch."Preventive medicine 36.5 (2003): 601-614.
Beatty, Paul C., and Gordon B. Willis. "Research synthesis: The practice of cognitive interviewing." Public Opinion Quarterly 71.2 (2007): 287-311.
Byers, Tim, et al. "The accuracy of parental reports of their children's intake of fruits and vegetables: validation of a food frequency questionnaire with serum levels of carotenoids and vitamins C, A, and E." Epidemiology 4.4 (1993): 350-355.
Contento, Isobel R. Nutrition education: linking research, theory, and practice. 3rd ed. Jones & Bartlett Learning, 2015.
de Leeuw, E., Borgers, N., & Smits, A. (2004). Pretesting questionnaires for children and adolescents. In S. Presser, J. M. Rothgeb, M. P. Cooper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.). Methods for testing and evaluating survey questionnaires. Hoboken,NJ: Wiley & Sons. 409-29.
Drennan, Jonathan. "Cognitive interviewing: verbal data in the design and pretesting of questionnaires." Journal of advanced nursing 42.1 (2003): 57-63.
Edmunds, L. D., and S. Ziebland. "Development and validation of the Day in the Life Questionnaire (DILQ) as a measure of fruit and vegetable questionnaire for 7–9 year
olds." Health Education Research 17.2 (2002): 211-220.
Hunsberger, Monica, et al. "Relative validation of Block Kids Food Screener for dietary assessment in children and adolescents." Maternal & child nutrition 11.2 (2015): 260-270.
Lim, Sue Sing, et al. "Validation of 2 Brief Fruit and Vegetable Assessment Instruments Among Third-Grade Students." Journal of nutrition education and behavior 47.5
(2015): 446-451.
Mayne, Susan, et al. “Resonance Raman spectrosopic evaluation of skin carotenoids as a biomarker of carotenoid status for human studies.” Archive of Biochemistry and
Biophysics. 539 (2013) 163-170.
Moore, G. F., et al. "Validation of a self-completion measure of breakfast foods, snacks and fruits and vegetables consumed by 9-to 11-year-old
schoolchildren." European journal of clinical nutrition 61.3 (2007): 420-430.
Penkilo, Monica, Goldy Chacko George, and Deanna M. Hoelscher. "Reproducibility of the School-Based Nutrition Monitoring Questionnaire among fourth-grade
students in Texas." Journal of nutrition education and behavior 40.1 (2008): 20-27.
Shafer, K., and B. Lohse. "How to conduct a cognitive interview: A nutrition education example." US Department of Agriculture, National Institute of Food and
Agriculture (2005).
Sharma, Manoj, and Rick Lingyak Petosa. Measurement and evaluation for health educators. 2nd ed. Jones & Bartlett Publishers, 2014.
Smith, Stephanie L., and Leslie Cunningham-Sabo. "Food choice, plate waste and nutrient intake of elementary-and middle-school students participating in the US
National School Lunch Program." Public health nutrition 17.6 (2014): 1255-1263.
Wallen, Victoria, et al. "Validation of a group-administered pictorial dietary recall with 9-to 11-year-old children." Journal of nutrition education and behavior 43.1
(2011): 50-54.
Cognitive interview protocol and
DILQ administration protocol available upon
request