Knowledge Translation&
Common Child and Youth Mental Disorders (CCYMD)
Offord Centre for Child Studies, November 26, 2015
Plan for Today
KT Refresher•KT & evidence-informed decision making•Synthesizing research evidence: the science & the art•Diffusion, dissemination or implementation?•Knowledge to Action Cycle•Implementation Science
KT & CCYM Disorders: Are We Ready for Prime Time?•SR, MA & PG: Do they meet international quality standards?•Knowledge repositories & tools: Users guide•Effective Knowledge Implementation Strategies?
Team
Alexa Bagnell Daniel Gorman Lynn Miller
Gail Bernstein Daphne Korczak Christopher Mushquash
Peter Braunberger Stanley Kutcher Amanda Newton
Jeffrey Bridge Paul Links Anne Rhodes
Melissa Brouwers Ellen Lipman Maureen Rice
Amy Cheung Katharina Manassis Robert Santos
Stephanie Duda Ian Manion Peter Szatmari
Jane Garland John McLennan Lehana Thabane
Common Child and Youth Mental Disorders (CCYMD)
Anxiety Disorders: Agoraphobia, generalized anxiety disorder, social phobia, specific phobia, panic disorder, separation anxiety disorder
Mood Disorders: Major depressive disorder, dysthymia
Disruptive Behaviour Disorders: Attention deficit hyperactivity disorder, oppositional defiant disorder, conduct disorder
What is Knowledge Translation?
CIHR Definition*:1. Process: Dynamic and iterative; includes synthesis, dissemination, exchange and ethically sound application of knowledge.
2.Three goals: improve the health of Canadians, provide more effective health services and products and strengthen the health-care system.
* CIHR. Knowledge translation and commercialization. (Updated Nov 17, 2015) http://www.cihr-irsc.gc.ca/e/29529.html
But Many Different Terms Out There…
McKibbon et al (2010). A cross-sectional study of the number & frequency of terms used to refer to knowledge translation in a body of health literature in 2006: A Tower of Babel? Implementation Science; 5:16.
• 100 different terms in 581 articles• CIHR definition used widely.
Integrated Knowledge Translation (iKT)*
1. KUs involved in all stages of research*: • increase relevance • facilitate uptake
2. Process: takes place within complex system of interactions between researchers and knowledge-users (KUs) which may vary in intensity, complexity and level of engagement depending on the nature of the research and findings, as well as the needs of the particular knowledge user.
* CIHR (2015). Guide to Knowledge Translation Planning at CIHR: Integrated and End-of-Grant Approaches. http://www.cihr-irsc.gc.ca/e/45321.html
* Gagnon ML. Moving knowledge to action through dissemination and exchange. Journal of Clinical Epidemiology, 2011
1. Process:• Systematic, transparent use of research evidence in
health practice and policy decision making
2. Principles:• Research evidence provides guidance not prescription• Research evidence doesn’t make decisions – people do• Primum non nocere: Above all, do no harm
What is Evidence-Informed Policy & Practice?
Evidence-informed Decision Making: 3 Pillars
First Step: Synthesize Research Knowledge
• Major focus/energy to date• Synthesis can:
1. Inform research agenda: • Identify research gaps & priorities • Provide rationale for new research
2. Facilitate research knowledge use in health services decisions:
• Strengthen health provider knowledge and intentions• Better care process decisions• Improve health outcomes
• Science and art
Targets of
Change
Research Evidence Pyramid: The Science
Why Synthesize Body of Knowledge?
• Ioannidis, PLoS 2005*: Need adequately powered, low bias evidence:
“ … most research questions are addressed by many teams, and it is misleading to emphasize the statistically significant findings of any single team. What matters is the totality of the evidence...”
• Ioannidis, JAMA 2005†: • 45/49 highly cited studies claim intervention effective• 16% contradicted by subsequent studies• 16% found bigger effects• 44% replicated• 24% remained largely unchallenged
* Ioannidis JPA. Why most published research findings are false. PLoS. 2005.
† Ioannidis JPA. Contradicted and Initially Stronger Effects in Highly Cited Clinical Research. JAMA. 2005.
Research Evidence Synthesis: The Art
1. What: Raw or digested?• Primary studies• SR/MA• PGs• Synopses & tools• Curated/quality assessed• Actionable recommendations
2. Who: Tailored and user friendly?• Practitioners• Patients• Managers• Policy-makers
3. How: Communication format/medium?1. e-Resources2. Reminders and updates3. Face-to-face4. Paper5. Combination6. Push, pull, push & pull
From Knowledge to Action
Knowledge Users*Knowledge Producers
* Policy-makers; Managers; Clinicians; Patients & Families
Knowledge To Action Cycle*
* Graham ID, et al. Lost in knowledge translation: Time for a map? The Journal of Continuing Education in the Health Professions. 2006.
Action!
Diffusion • Passive spread of evidence
Dissemination
• Active, tailored and planned distribution of evidence to target groups to encourage adoption of innovations
Implementation
• Active efforts to put evidence into practice
Knowledge To Action Cycle*
* Graham ID, et al. Lost in knowledge translation: Time for a map? The Journal of Continuing Education in the Health Professions. 2006.
Implementation Science
“Scientific study of methods to promote the systematic uptake of research findings & other evidence-based practices into routine practice to improve the quality and effectiveness of health services and care”*
• Identify barriers and facilitators to practice & policy change• Create interventions to promote research uptake• Theories, models & frameworks
* Eccles, MP & Mittman BS. Welcome to Implementation Science. Implementation Science, 2006. * Nilsen P. Making sense of implementation theories, models and frameworks. Implementation Science, 2015.
Theories, Models & Frameworks*
Three aims: • Describe translation process• Understand or explain what influences implementation
outcomes• Evaluate implementation
Theories • Clear explanation of how and why specific relationships lead to specific events
Models • Simplification of a phenomenon• Descriptive rather than explanatory
Frameworks • Describe empirical phenomena by fitting them into categories
• Not explanatory
Common Sense
• “Informal” theories
* Nilsen P. Making sense of implementation theories, models and frameworks. Implementation Science, 2015.
Consolidated Framework for Advancing Implementation
Science (CFIR)
• Damschroder, 2009*:• Identified 19 theories/models/frameworks• Created CFIR • Overarching typology to promote implementation theory
development and verification about what works where and why across multiple contexts
*Damschroder LJ, et al. Fostering implementation of health services research findings into practice: Consolidated framework for advancing implementation science. Implementation Science, 2009.
CFIR: Domains & Constructs
Intervention Characteristic
s
Outer Setting
Inner Setting
Characteristics of
Individuals Involved
Process of Implementation
• Intervention Source• Evidence Strength
& Quality• Relative Advantage• Adaptability• Trialability• Complexity• Design Quality &
Packaging• Cost
• Patient Needs & Resources
• Cosmopolitanism• Peer Pressure• External Policies
& Incentives
• Structural Characteristics• Networks &
Communications• Culture• Implementation Climate: Tension for change Compatibility Relative Priority Organizational Incentives &
Rewards Goals & Feedback Learning Climate• Readiness for
Implementation: Leadership Engagement Available Resources Access to
Information & Knowledge
• Knowledge & Beliefs about the Intervention
• Self-efficacy• Individual Stage of
Change• Individual
Identification with Organization
• Other Personal Attributes
• Planning• Engaging: Opinion Leaders Formally
Appointed Internal
Implement- ation Leaders Champions External Change Agents• Executing• Reflecting & Evaluating
Effective Knowledge Implementation Interventions
• Implementation processes combined with high quality research knowledge
• Grimshaw et al, 2012*: What we know so farProfessional behaviour changePolicy makers and senior health service managers
* Grimshaw J, et al. Knowledge translation of research findings. Implementation Science, 2012.
Effective Knowledge Implementation Strategies
Professional Behaviour Change Strategies•Printed educational materials•Educational meetings•Educational outreach•Local Opinion Leaders•Audit and Feedback•Computerized Reminders•Tailored Interventions
Policy-makers and Senior Health Service Managers•? but many innovative approaches developed and worthy of testing
KT & CCYM Disorders: Are We Ready for Prime Time?
1. Research Synthesis: SR/MA & PG Quality?2. Dissemination: Knowledge Repositories & Tools
– Coverage & Quality?3. Implementation: Effective Strategies?
Do Systematic Reviews and Meta-analyses About CCYM Disorders Meet
International Quality Standards?
• Cochrane & PRISMA Systematic review methods• Research librarian created & conducted search• Two independent reviewers• Inclusion criteria:
• Systematic review or meta-analysis• Prevention or treatment• Anxiety, depression, suicide related behaviors• Aged ≤18 years• English language• 2000 – 2012
PRISMA Flow Diagram
Unique Records Identified & Screened
Eligible Reviews
35 14Anxiet
yDepressi
onSRB
20 16Anxiety
& Depressi
on
4194
85
Quality Assessment Methods
AMSTAR*• Assessing the Methodological Quality of Systematic
Reviews• 11 items• Minimum quality defined as:
• Systematic reviews: 5/9• Meta-analyses: 6/11
• 2 raters; Disagreements resolved by consensus
*Shea BJ, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology, 2009.
How Many Eligible Reviews Meet Minimum Quality
Standards?
Disorder (n)
% Meet Minimum Quality Standards
Anxiety (20) 40.0
Depression (35) 60.0
Anxiety & Depression (16) 37.5
Suicide Related Behaviors (14) 50.0
Total Reviews (85) 49.4
AMSTAR Criteria Failure Rate
Risk of Bias Criteria Failure Rate (%)
Research Question & Inclusion Criteria Stated 5.9
Duplicate Study Selection & Data Extraction 74.1
Comprehensive Literature Search Performed 44.7
Grey Literature & Other Languages Considered 61.2
Included & Excluded Studies Listed 56.5
Characteristics of Included Studies Provided 16.5
Study Quality Assessed & Documented 67.1
Study Quality Considered in Conclusions 62.4
Appropriate Meta-analysis Methods* 9.5
Publication Bias Assessed* 47.6
Funding Source & Conflict of Interest Reported 68.2
* Data based on reviews containing meta-analyses only (n=42)
Did Introduction of AMSTAR in 2007 Improve Review Quality?
No significant difference in AMSTAR scores 2007-2009 compared to 2010-2012; t(54)= -1.08, p = 0.286.
Does Journal Impact Factor Predict Review Quality?
Impact Factor
nMean
AMSTAR Score (sd)
≤ 3.000 29 4.64 (2.23)
3.001 – 4.000
14 4.75 (2.05)
4.001 – 6.000
16 5.75 (2.81)
6.001 – 8.000
19 7.47 (3.47)
≥ 8.001 3 6.83 (1.89)
Not Reported 4 4.00 (2.48)Correlation between AMSTAR Score & Impact Factor: r = 0.25, p<0.05
Do Practice Guideline Development Methods Meet International Quality
Standards?*
• Cochrane and PRISMA systematic review methods• Research librarian created & conducted search• Two independent reviewersStep 1: Find CYMH PGs (published or updated 2009-2014)
• Journals (4) of leading CYMH professional organizations• National Guideline Clearinghouse web-site• Websites of organizations who produce PGs: NICE, SIGN,
USPSTFStep 2: Find CYMH development methods
• Eligible PGs reviewed to identify cited or associated development methods
*Bennett K, et al. Practitioner Review: On the trustworthiness of clinical practice guidelines: A systematic review of the quality of methods used to develop guidelines in child and youth mental health. Journal of Child Psychology and Psychiatry (Forthcoming).
PRISMA Flow Diagram
PGs
PG Development Methods
PGs Identified
Unique PGs Screened
Eligible PGs
PG Development Methods Screened
Eligible PG Development Methods Sets
PG Quality Rating Methods
AGREE II*: • Appraisal of Guidelines for Research and Evaluation• Validated PG quality assessment tool• 6 domains
Scoring Options:
*Brouwers M, et al. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. CMAJ, 2010.
Score
Interpretation
0 Domain/standard not mentioned
1Domain/standard optional - low confidence that the standard would be adhered to by PG developers
2Domain/standard optional - moderate confidence that the standard would be adhered to by PG developers
3Domain/standard mandatory - high confidence that the standard would be adhered to by PG developers
AGREE II Ratings
PG Development Methods
AGREE II DomainScore
Frequency
Scope & Purpose
Stakeholder
Involvement
Rigour of Develop-
ment
Clarity of
Present-ion
Applica-abiilty
Editorial Indepen-
dence0 1 2 3
NICE 3 3 3 3 3 3 0 0 0 6
SIGN 3 3 3 3 3 3 0 0 0 6
USPSTF 3 2 3 3 1 3 0 1 1 4
AAP 2 2 1 3 1 2 0 2 3 1
AACAP 2 1 1 1 0 1 1 4 1 0
AACAP = American Academy of Child and Adolescent PsychiatryAAP = American Academy of PediatricsUSPSTF = U.S. Preventive Services Task ForceNICE = National Institute for Health and Care ExcellenceSIGN = Scottish Intercollegiate Guidelines Network
0 = not mentioned 1 = optional - low confidence 2 = optional - moderate confidence 3 = mandatory - high confidence
Summary
Source
Development Methods
Meet Internation
al Standards?
Percent Of Available
PGsRepresente
d
PG Development Organizations:
31.5%
NICE & SIGN All 22.9%
USPSTF Most 8.6%
Specialty Societies: 21.4%
AACAP No 17.1%
AAP No 4.3%
Other or not eligible: ? 47.1%
68.5%
Conclusions
1. Up to 69% of PGs developed using methods that don’t align with AGREE II
2. Do professional specialty societies have the resources required to produce high quality PGs?
3. Quality of individual PGs needs to be assessed (underway)
4. Need strategies to:• Guide users to high quality PGs quickly• Facilitate adherence by PG developers to quality standards• Avoid harm and wasted resources
Dissemination: e-Resources/Technologies
Repositories and Clearinghouses:• What’s out there?• How good are they?• Do they meet user needs?• Pilot study findings
Research Repositories: ‘Best in Class’
• Rx for Change • EvidenceUpDates• HealthEvidence.org • HealthSystemsEvidence.org • Cochrane Database of Systematic Reviews
‘Best In Class’ Knowledge Repositories
Rx for Change EvidenceUpdates HealthEvidence.orgHealthSystemsEvidence
.org
Cochrane Database of Systematic
Reviews
Source University of Ottawa Jeremy Grimshaw
McMaster University Brian Haynes
McMaster UniversityMaureen Dobbins
McMaster Health ForumJohn Lavis
The Cochrane Collaboration
Purpose Behaviour change strategies for drug
prescribing and health technology use
Evidence-based medicine
Evidence-based public health decision-
making
Evidence-based health system strategies
Evidence-based medicine
Target Audience
Health Care Professionals & Policy-
makersPhysicians
Public Health Practitioners/
Managers/Policy-makers
Health System Policy-makers/Stakeholders
Health Care Decision Makers (Clinicians,
Patients, Researchers, Policy-makers, etc.)
Coverage of CCYMD† 0 61/?
2.5%(109/4,401)
80/?0.34%
(31/9,156)
Content•Original• Synopses
Original (SR)&
Synopses (SR)Original (PS, SR) Original (SR) Original (PS, SR) Original (SR)
Quality Appraised
X
Quality Appraisal Method
AMSTAREvidence Update Quality
Inclusion CriteriaOxman OQAQ tool* AMSTAR N/A
Searchable Database
Updates/Alerts Service
X
(Tailored)
(Tailored)
(Tailored)X
* OQAQ = Overview Quality Assessment Questionnaire (Oxman & Guyatt, Journal of Clinical Epidemiology, 1991)† # of hits when searching for ‘child and adolescent mental health’
CCYMD Repositories
• SAMHSA National Registry of Evidence-based Programs and Practices (NREPP)
• California Evidence-based Clearinghouse for Child Welfare (CEBC)
• Child Trends LINKS database• Canadian Best Practices Portal• What Works Clearinghouse• Ontario Centre of Excellence for CYMH Evidence
In-Sight and Policy Ready Papers• U-Mind
CCYMD Knowledge Repositories 1NREPP CEBC
What Works/LINKS database
Canadian Best Practices Portal
Source Substance Abuse & Mental Health Services
Administration (SAMHSA)
California Dept. of Social Services
Child Trends (US-based NPO)
Public Health Agency of Canada
Purpose Evidence-based Mental Health & Substance Abuse Interventions
Evidence-based Practices for Child Welfare System
Evidence-based programs relevant to
Child Health & Education
Successful Disease Prevention & Health
Promotion Interventions
Target Audience
Public
State-wide agencies, Counties, Public &
Private Organizations & Individuals
?Health Professionals & Public Health Decision-
makers
Coverage of CCYMD 5.8%(22/378)
70/? 1,138/? 14/?
Content•Original•Synopses
Synopses Synopses Synopses Synopses
Quality Appraised X X
Quality AppraisalMethod NREPP Quality of
Research Criteria
CEBC Scientific Rating Scale & CEBC
Measurement Tools Rating Scale
N/A N/A
Searchable Database
Updates/Alerts Service
(General)
(General)
(General)
X
CCYMD Knowledge Repositories 2
What Works Clearing-house
Evidence In-Sight Policy Ready Papers U-Mind
SourceInstitute of Education
Sciences, U.S. Dept. of Education
Ontario Centre of Excellence for Child & Youth Mental Health
Ontario Centre of Excellence for Child & Youth Mental Health
Kids’ Health Links Foundation &
Lutherwood Institute for Children and Youth
Mental Health
PurposeEvidence-based
EducationEvidence-based Practice
in CYMHEvidence-based Policy in
CYMH
CYMH Professional Resources & Practical
Tools (Content Shared by Members?)
Target AudienceEducators
Community Agency CYMH Professionals
Policy-makers & Decision-makers
CYMH Professionals
Coverage of CCYMD0
100%(60/60)
100%(16/16)
100%
Content•Original•Synopses
Synopses Synopses
(Rapid Reviews Upon Request)
Synopses Neither
(see Purpose)
Quality Appraised X X X
Quality AppraisalMethod
WWC Group Design Standards
N/A N/A N/A
Searchable Database X
Updates/Alerts Service
X X X
(General)
Summary
• What is good enough?• Who are the users?• What are their needs?
Effective Knowledge Implementation Interventions: 2
SRs So FarNovins et al (2013)*:
• 73 eligible articles (16 RCTS)based on 44 host studies • Inner context factors:
• Strongest evidence: fidelity monitoring & supervision• Less evidence for organizational climate/culture but better
sustainment and youth outcomes
• Outer context factors:• Strongest evidence: training and use of special technologies
Barwick et al (2012)†:• 12 intervention studies• Poor quality; self report of behavior change
*Novins DK, et al. Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. JAACAP, 2013.† Barwick MA, et al. Knowledge translation efforts in child and youth mental health: A systematic review. Journal of Evidence-based Social Work, 2012.
What Are We Doing Re Implementation?
1. CIHR Dissemination Grant:• Disseminating Child and Youth Mental Health Practice
Guidelines: The Development of a User-Informed, Social Media Integrated, Mobile Website
2. Tailored PG implementation tool
Questions & Thank-you
Top Related