KIMBERLY DEBOTHBALL STATE UNIVERSITY

APRIL 7 , 2014

NASP Domain 9: Research and Program Evaluation

Research & Program Evaluation

School psychologists have knowledge of research design, statistics,

measurement, varied data collection and analysis techniques, and program

evaluation sufficient for understanding research and

interpreting data in applied settings.

Overview

Service Delivery ModelResearch TrendsMeasurement & Statistical AnalysisSingle-Subject Research DesignResearch to PracticeApplication to RTIProgram Evaluation

Model for Services

Research Trends in School Psychology

Analysis of Research Designs in Published Journals Experimental/intervention research articles (n= 929) School Psychology Review, Psychology in the Schools,

School Psychology Quarterly, Journal of School Psychology

Measurement vs. intervention research Decrease in general education research Mixed methods approachNarrative Correlation

alDescriptive Casual-

Experimental

Casual-Comparativ

e

Meta-analytic

40% 29% 16% 8% 4% 2%

(Bliss, Skinner, Hautau & Carroli, 2008; Powell, Mihalas, Onwuegbuzer, Suldo & Daley, 2008)

(Swaminathan & Rogers, 2007)

Measurement & Statistical Analysis

Uncertainty of measured behaviorReliability of observations

Interrater agreement Duration Generalizability

Data analysis Meta-analysis Hierarchical Linear Modeling Structural Equations Modeling

Single-Subject Research Design

Individual StudentsPopularity- Evidence Based Intervention

MovementManipulate VariablesInternal Validity ThreatsTask Force for Evidence Based Interventions

in School Psychology Sponsored by Division 16 by APA Coding System for Group & Single Subject Design Classify Evidence-Based Interventions

(Christ, 2007; Jenson, Clark, Kircher & Kristjansson, 2007; Parker, Cryer & Byrns, 2006)

How to Translate Research into Practice

Randomized controlled experiments in school settingsGold standard- true experimental designLimitations:

ResourcesAcceptabilitySocial ValidityFidelity

Quasi-Experimental DesignRole of the School Psychologist

ResearchData-Based Decision Making Intervention PlanningTeacher SupportExample- Reading Program

(NASP, 2005; Bliss, Skinner, Hautau & Carroli, 2008; Christenson, Carlson & Valdez, 2002)

(Skinner, Mccleary, Skolits, Poncy & Cates, 2013)

Application to Response to Intervention

Standardized procedures- universal screenings

Reliable and valid measurementIntervention integrityThreats to internal & external validitySingle subject research designNeed for research

Learning strategies Interventions Procedures

Program Evaluation

Utilize a comprehensive approach Acceptability of program Integrity of implementation Impact/effectiveness of program

CollaborationAccountability

Individual Group System

(McDougal, Nastasi & Chafouleas, 2005; NASP, 2010)

Literature Review Method

Searches for peer-reviewed articles were conducted using PsychINFO and Google Scholar with the following keywords: Research design AND school psychology Program evaluation AND school psychology

Results yielded 436 articles between 2000-2014

References

Bliss, S. L., Skinner, C. H., Hautau, B., & Carroli, E. E. (2008). Articles published in four school psychology journals from 2000 to 2005: An analysis of experimental/intervention research. Psychology In The Schools, 45(6), 483-498.

Christ, T. J. (2007). Experimental control and threats to internal validity of concurrent and nonconcurrent multiple baseline designs. Psychology In The Schools, 44(5), 451-459.

Christenson, S. L., Carlson, C., & Valdez, C. R. (2002). Evidence-based interventions in school psychology: Opportunities, challenges, and cautions. School Psychology Quarterly, 17(4), 466-474.

Jenson, W. R., Clark, E., Kircher, J. C., & Kristjansson, S. D. (2007). Statistical reform: Evidence-based practice, meta-analyses, and single subject designs. Psychology In The Schools, 44(5), 483-493.

McDougal, J. L., Nastasi, B. K., & Chafouleas, S. M. (2005). Bringing research into practice to intervene with young behaviorally challenging students in public school settings: Evaluation of the behavior consultation team (BCT) project. Psychology In The Schools, 42(5), 537-551.

National Association of School Psychologists. (2010). Model for Comprehensive and Integrated School Psychological Services. Bethesda, MD: Author.

National Association of School Psychologists. (2005). Prevention and intervention research in the schools (Position Statement). Bethesda, MD: Author.

Parker, R. I., Cryer, J., & Byrns, G. (2006). Controlling baseline trend in single-case research. School Psychology Quarterly, 21(4), 418-444.

Powell, H., Mihalas, S., Onwuegbuzie, A. J., Suldo, S., & Daley, C. E. (2008). Mixed methods research in school psychology: A mixed methods investigation of trends in the literature. Psychology In The Schools, 45(4), 291-309.

Skinner, C. H., Mccleary, D. F., Skolits, G. L., Poncy, B. C., & Cates, G. L. (2013). Emerging opportunities for school psychologists to enhance our remediation procedure evidence base as we apply response to intervention. Psychology In The Schools, 50(3), 272-289.

Swaminathan, H., & Rogers, H. (2007). Statistical reform in school psychology research: A synthesis. Psychology In The Schools, 44(5), 543-549.