Enhanced Implementation of CDC Guidelines for School Tobacco Programs:

Results of a Statewide Evaluation

Shelly A. Greller, MSWisconsin Department of Public Instruction

D. Paul Moberg, Ph.DUniversity of Wisconsin – Madison

Acknowledgements

• Wisconsin Department of Public Instruction– John Hisgen

– Douglas White

• University of WI – Monitoring & Evaluation Program (www.medsch.wisc.edu/mep)– David Ahrens

– Ann Christiansen

– Amy Anderson

– Patrick Remington, P.I.

• Funded by the Wisconsin Tobacco Control Board

Participants should be able to:

• Describe WI statewide school tobacco grant program

• Describe CDC Guidelines for School Health Programs to Prevent Tobacco Use and Addiction

• Identify effective program resources and strategies

• Understand & apply results of WI school tobacco program to enhance implementation of CDC Guidelines

• Apply assessment & evaluation tool to operationalize CDC Guidelines

Background

• Master Settlement Agreement led to creation of WI Tobacco Control Board (WTCB)

• WTCB developed comprehensive strategic plan for tobacco control– One focal area was youth tobacco prevalence– Youth goal: By 2005, tobacco use among middle &

high school age youth will decline by 20%

• School-based programs funded for $1,250,000 by competitive process through DPI

Background

• DPI stated purpose for school-based tobacco control programs– “Create or expand upon strategies identified as most

promising or effective in reducing or eliminating youth tobacco use”

• Operationally, discussion focused on increased implementation of CDC Guidelines

Monitoring and Evaluation

• Monitoring and Evaluation Program (MEP) contracted to perform monitoring/evaluation functions for WTCB

• MEP comprised of– UW – Comprehensive Cancer Center – UW – Extension– Center for Health Policy and Program Evaluation

• MEP collaborated with DPI on evaluation of school-based tobacco control programs

CDC Guidelines

• Guidelines for School Health Programs to Prevent Tobacco Use and Addiction (MMWR, 1994)– Summarizes school based strategies “most likely to

be effective in preventing tobacco use by youth”– Developed by CDC in collaboration with experts– In-depth review of research, theory & current

practice

Impact of Guidelines

• Rohde et al. (2001 in MMWR)

– High or medium implementation of Guidelines in Oregon associated with significantly greater decline in 8th graders’ 30-day smoking prevalence

Impact of Guidelines

• Rohrbach et al. (2002 Presentation at Society for Prevention Research-SPR)

– 2 waves of student data from sample of schools– Indexed Guideline Implementation based on teacher

surveys– Most schools did not fully implement

comprehensive program of evidence-based strategies

– Guideline implementation significantly related to smoking prevalence - quit attempts - negative expectations/attitudes regarding tobacco

Impact of Guidelines

• Hallfors & Godette (2002, HER)

– Dept. of Education has recent statement of “Principles of Effectiveness” for prevention programs

– Surveyed 104 School Districts in 12 states– Only 19% of school districts are implementing

research-based prevention curricula with “fidelity”

Goal of Our Evaluation

• Focus on institutional change in implementation of the guidelines in schools with infusion of targeted funding

• Not on impact of implementation of Guidelines on student outcomes

Methods of Our Evaluation

2001-’02 2002-’03

0 $$$$ 0 $$ 0

0 0 [0?]

0 = Self Assessment Form –School Building

$$$$ = Funded District

DesignSample: Applicant School Buildings/Districts

Measures – Self Assessment

• Each building in applicant district completed checklist operationalizing CDC Guidelines– Operationalizes CDC Guidelines with 58 questions– Original purpose was self assessment and planning– Program evaluation secondary

• Repeated at end of year 1 and year 2– Funded districts submitted as part of year end report– Unfunded districts mailed as survey at end of year 1

Scaling/Data Reduction

• Exploratory Principal Components factor analysis indicated most items within areas of the guidelines were correlated

• We created additive indices for each of the 7 areas

• Scaling0 = No

1 = Somewhat

2 = Yes

Scale Validation

• 2001-2002 School Health Education Profile (SHEP) oversampled to include all funded schools

• Reports from SHEP will be correlated with reports from schools

• Compared baseline data of funded schools to full SHEP sample– How representative are they?

Scale ConstructionScale # Items Mean S.D. Alpha

School Policies 17 1.55 0.33 .82

Curriculum 17 1.29 0.48 .95

Instruction 6 0.98 0.42 .59

Training 4 0.54 0.58 .77

Family & community involvement

7 0.66 0.47 .79

Cessation Services 3 0.41 0.52 .62

Evaluation 4 0.41 0.49 .84Note: Reported for baseline data, n=320. Means in terms of original metric: 0=No, 1=Somewhat, 2=Yes

Sample Size & Response Rate

Assessment Not funded (Comparison)

Funded Total

Baseline 154 180 334

Follow-Up* 61 (40%) 155 (86%) 216 (65%)

*27 additional buildings submitted follow-ups without baselines

Buildings represent 98 Districts (42 funded) and 11 Consortia (4 funded)

Mean Change During Year 1Variable Baseline One Year Difference T-Statistic

Policy

Funded 1.54 1.72 .177 7.15**

Unfunded 1.66 1.78 .125 3.29*

Curriculum

Funded 1.19 1.49 .296 7.87**

Unfunded 1.49 1.65 .161 3.03*

Instruction

Funded 0.99 1.28 .287 8.58**

Unfunded 1.49 1.65 .134 2.32**P < .05 **P < .001

Mean Change During Year 1Variable Baseline One Year Difference T-Statistic

Training

Funded 0.60 0.99 .390 7.23**

Unfunded 0.63 0.85 .215 2.49*

Family / Community Involvement

Funded 0.68 1.04 .365 9.38**

Unfunded 0.70 0.89 .191 2.42*

Cessation

Funded 0.42 0.90 .481 9.08**

Unfunded 0.45 0.67 .218 2.68**P < .05 **P < .001

Mean Change During Year 1

Variable Baseline One Year Difference T-Statistic

Evaluation

Funded 0.45 0.88 .422 7.55**

Unfunded 0.50 0.79 .289 2.74*

Overall

Funded 1.08 1.38 .294 11.5**

Unfunded 1.23 1.41 .179 4.5**

*P < .05 **P < .001

Regression Results

Regression analysis covarying

baseline value found:

• Baseline covariate always a sig. predictor of one year implementation measure

• Significant effects of funding on– Training: B = .155 (.084)– Family / community involvement: B = .162 (.072)– Cessation: B = .25 (.09)

Percentage of Funded SchoolsReporting Positive Change

Any Positive Change

+.25 SD Pos. Change

# Schools Reporting

Policy 66.9 % 55.0 % 151

Instruction 66.2 64.1 142

Curriculum 66.7 53.9 141

Training 61.9 61.9 134

Parent/community 74.6 61.2 126

Cessation 63.3 63.3 147

Evaluation 59.7 59.7 134

Changed on any 90.3 89.0 155

Funded Schools Prohibit Tobacco Use…

0 20 40 60 80 100

Staff/Visitor - in school vehicles

Staff/Visitor - at functions off schoolproperty

Students - at functions off schoolproperty

Students - in facilities

Baseline Change to Follow-up

Funded School Policies…

0 20 40 60 80 100

clear rationale focusing on health risk

prohibit tobacco advertising

provide swift, consistent, equitableenforcement

consistently & equitably enforced

communicated to students, staff, parents,visitors by varitey of means

Baseline Change to Follow-up

Funded Schools Tobacco Instruction…

0 20 40 60 80 100

to teacher/staff in curriculum theory &model

integrated into broader health program

provided in every grade in school

in avoiding tobacco use

Baseline Change to Follow-up

Rather Than Punitive – Funded Schools…

0 20 40 60 80 100

consistently offerstudents help

offer preventioneducation & access

to cessationprograms

Baseline Change to Follow-up

Funded Schools Teach Developmentally Appropriate

0 20 40 60 80 100

short & long term physical consequences

short & long term social consequences

accurate social norms re: tobacco

skills for resisting social influences

general personal & social skills

Baseline Change to Follow-up

Funded Schools Teach Developmentally Appropriate

0 20 40 60 80 100

health benefits tobacco-free environment

skills to encourage others not to use

skills to identify & counter tobacco ads

skills to cope with parent/family use

advocacy skills i.e. request smoke-free

Baseline Change to Follow-up

Does Funded School..

0 20 40 60 80 100

involve parent/family in policydevelopment

involve youth in policy development

promote home discussions thruhomework

encourage families to reinforce

regularly assess effectiveness of policies

Baseline Change to Follow-up

Does Funded School..

0 20 40 60 80 100

regularly assess efectiveness ofcurriculum

provide parent education on smokingissues

participate in local coalition/ partnership

offer cessation services for students

offer cessation for staff or refer tocommunity programs

Baseline Change to Follow-up

Variation by Level – Funded Schools• Elementary schools (57 schools)

– High on baseline family & community involvement– Lowest on evaluation

• Middle schools (52 schools)– Highest overall score at baseline, especially in training and

curriculum

• High schools (30 schools)– More change (than elementary & middle schools) on policy,

training, parent & community involvement & cessation– Highest initially on cessation

• Combinations (16 schools)– Highest level of change overall: high change on policy,

curriculum, and instruction

Limitations and Issues

• Tool originally intended as self assessment– Not a research tool– Validity/reliability to be established

• SHEP link will help – in progress

• Self Report from school staff (who want funding to continue) without validation

• Low response rate from comparison group

• No student outcomes

Conclusions

• Baseline implementation of Guidelines highest in areas of– Policy– Curriculum

• Implementation of Guidelines lowest in– Cessation– Evaluation– Training– Family / community involvement

Conclusions

• Most gain with funding in – Cessation– Evaluation– Training– Family / community involvement

• Unfunded schools also report gain in all areas--but significantly less in several areas

• The tool is promising to assess school tobacco control programs both for planning & evaluation