Evidence-Based Practice in the

Applied Social Sciences

by Robin Paynter

Vested Communities

Academic Researchers

Government Agencies

Practitionersin the field

Service Users

Insurance CompaniesEvidenceBased

Practice

Professional Organizations

Governmental Agencies

United States. Department of Education. (US DOE):No Child Left Behind Act of 2001 requires the use of relevant scientifically based research by educators. US DOE created the What Works Clearinghouse (see further information below)

Oregon. Senate Bill 267

“A unique bipartisan bill passed by the Oregon Legislature in 2003 requiring key state agencies (e.g., Department of Corrections, Oregon Youth Authority, Oregon Criminal Justice Commission, the State Commission on Children and Families and the Department of Human Services*) to budget and implement “evidence-based programming” at incremental levels over a five year period; the legislation defined an evidence based program as “incorporates significant and relevant practices based on scientifically based research and is cost effective.”

From http://www.ssw.pdx.edu/focus/ebp/oregon.php. Accessed 04/01/2009.

*Emphasis added

Medical vs. Social Sciences ResearchMedline = 17,000,000+ records----------------------------------------------------------------------------

PsycINFO = 2,600,000+ recordsERIC = 1,282,000+ recordsSocial Services Abstracts = 149,000+Criminal Justice Abstracts = 99,200+----------------------------------------------------------------------------

▪ EVP requires a lot of research at the bottom of the pyramid to do higher level analyses - social science researchers have to go down the pyramid to find evidence as it is usually not available as RCTs, Meta-analyses, or systematic reviews

▪ Social science research characteristics that effect evidence-based research:

– small sample sizes– higher percentage of qualitative

research, especially in some disciplines

Research Skills & Strategies

Model Query:

P = Patient Population

I = Intervention

C = Comparison Intervention

O = Outcomes

Application to SocialScience Research:

▪ PICO usually doesn’t work

▪ Find evidence farther down the pyramid

▪ Search Multiple Databases

Example: P= Veterans with PTSD I= Eye Movement Desensitization and

Reprocessing (EMDR) therapy C= ‘Classic’ Desensitization therapy O= Reduce severity of symptoms

Formulating Questions / Doing EBM Researchhttp://www.cebm.net/index.aspx?o=1036

Researcher Tool-Kit http://www.ebm.med.ualberta.ca/ebm.htm

Multidisciplinary DatabasesCampbell Collaboration: C2 SPECTRDescription: “The Campbell Collaboration’s Sociological, Psychological, Educational, and Criminological Trials Register (C2-

SPECTR) contains over 11,600 citations to trials or possible trials. Studies are included in the register if they can be classified as a randomized control trial (RCT); cluster, place, or group randomized control trial (CRT), a possible randomized control trial (CCT), or a Systematic Review (WPH).” (Description from:

http://geb9101.gse.upenn.edu/RISIMAGES/c2spectrsearch.pdf Accessed July 10, 2008)Availability: Free database: http://geb9101.gse.upenn.edu/RIS/RISWEB.ISASearching Tips: Keyword searching only. Citations gathered largely from other bibliographic databases, e.g. ERIC, Sociological

Abstracts, Criminal Justice Abstracts, etc.

Campbell Collaboration: Campbell Library of Systematic ReviewsDescription: “The Campbell Library of Systematic Reviews provides free online access to systematic reviews in the areas of

education, criminal justice and social welfare. The library is a peer-reviewed source of reliable evidence of the effects of interventions.” (Description from http://www.campbellcollaboration.org/frontend.aspx. Accessed August 12, 2008)

Availability: Free database: http://www.campbellcollaboration.org/frontend.aspxSearching Tips: Reviews are listed alphabetically and also by area: Education, Crime and Justice, and Social Welfare.

Database of Abstracts of Review Effects (DARE)Description: Contains 5000+ abstracts of quality assessed and critically appraised systematic reviews. The database focuses

on the effects of interventions used in health and social care. Reviews identified by searching Medline, PsycINFO, Cochrane, etc.

Availability: Free database: http://www.crd.york.ac.uk/crdweb/Searching Tips: Search MeSH and keyword terms to increase search inclusiveness (e.g., ‘myocardial infarction’ retrieves

different results to searching ‘heart attack’). DARE includes records from many sources, not all of which use MeSH.

Turning Research Into Practice (TRIP)Description: Metasearch engine of evidence-based practice databases and websites, including: DARE, PubMed, Cochrane, EBP

journals, and more.Availability: Free database: http://www.tripdatabase.com/index.html

Psychology ResourcesPsycINFO

Description: PsycINFO provides citations and indexing to the published psychology literature.

Availability: Subscription database

URL: http://www.apa.org/psycinfo/

Searching TipsDESCRIPTORS: LIMITS: Evidence Based Practice LiteratureTreatment Effectiveness Review Evaluation SystematicMental Health Program Review Evaluation Meta AnalysisTreatment OutcomesPsychotherapeutic OutcomesClinical TrialsTreatment Guidelines

Medline/PubMed

Description: Comprehensive database covering research literature in all areas of medicine.

Availability: Subscription Medline database: widely available via commercial vendors, for example: CSA, EBSCO, FirstSearch, Ovid, etc

URL: Free PubMed database:http://www.ncbi.nlm.nih.gov/sites/entrez?

db=PubMed

Searching TipsDESCRIPTORS: LIMITS:Evidence-Based Medicine GuidelineDecision Support Techniques Meta-

AnalysisPractice Guidelines

RandomizedMeta-Analysis

ControlledRandomized Controlled Trials Trial

Review

Psychology Cont’d.Cochrane Library

Description: The Cochrane Library produces and disseminates full text systematic reviews of healthcare interventions and promotes the search for evidence in the form of clinical trials and other studies of interventions.

Availability: Subscription database

URL: http://www.cochrane.org/reviews/clibaccess.htm

Searching Tips:

DESCRIPTORS:Evidence-Based MedicineDecision Support TechniquesPractice GuidelinesMeta-AnalysisRandomized Controlled TrialsTreatment Outcome

Keyword searching: best results typically achieved by using ‘Title, Abstract, or Keywords’ search. Uses MeSH (Medical Subject Headings)

Manuals, Guidelines, etcEssential Evidence Plus (formerly InfoPOEMs/InfoRetriever)

Description: “The only database system of filtered, synopsized, evidence-based information, this integrated search engine allows you to simultaneously search, via keyword, multiple databases… EBM Guidelines is a unique, concise and easy-to-use collection of clinical guidelines for primary care combined with the best available evidence. EBM Guidelines includes 1,000 concise primary care practice guidelines, 3,000+ high quality evidence summaries and a library of 1,000 photographs and images.” Description from: https://www.essentialevidenceplus.com/product/features_ir.cfm Accessed August 14, 2008.

Availability: Subscription database

URL: http://www.essentialevidenceplus.com/index.cfm

Searching Tips: Browsing of subject guideline categories (e.g., ‘Psychiatric and Substance Abuse’) and keyword searching: within a subject guideline area or across the entire database.

KEYWORDS:DSM disorder name (e.g., post-traumatic stress disorder or PTSD)Drug name (e.g., Lexapro)Treatment (e.g., cognitive behavioral therapy or CBT)

Manuals, Guidelines, etc Cont’d.National Guideline ClearinghouseDescription: "A public resource for evidence-based clinical practice guidelines. NGC is an

initiative of the Agency for Healthcare Research and Quality (AHRQ), U.S. Dept. of Health and Human Services." While the primary focus of this site is currently medical guidelines, there are an increasing number of mental health/psychotherapy guidelines available.

Availability: Free database

URL: http://www.guideline.gov/

Searching Tips: Mental health guidelines listing (184 guidelines as of this writing): http://www.guideline.gov/browse/browsemode.aspx?node=16845&type=1

The detailed search page offers additional ways to search, however researchers can easily and quickly specify too many elements and retrieve zero results: http://www.guideline.gov/search/detailedsearch.aspx

American Psychiatric Association Practice GuidelinesDescription: Evidence-based recommendations for the assessment and treatment of

psychiatric disorders.Availability: Free databaseURL: http://www.psych.org/psych_pract/treatg/pg/prac_guide.cfmSearching Tips: Guidelines are arranged alphabetically by disorder (Diagnostic and

Statistical Manual of Mental Disorders classification). Some links lead to subscription content on http://www.psychiatryonline.com.

Social Work ResourcesSocial Services Abstracts

Description: Citations to research focused on social work, human services, social welfare, social policy, and community development. Major areas of coverage: community and mental health services; crisis intervention; family and social welfare; gerontology; poverty and homelessness; health policy; addiction; social work.

Availability: Subscription database

URL: http://www.csa.com/factsheets/ssa-set-c.php

Searching Tips:

DESCRIPTORS KEYWORDS:Evidence-based Systematic

Review practiceTreatment Outcomes Meta-analysis

California Evidence-Based Clearinghouse for Child Welfare

Description: “The California Evidence-Based Clearinghouse for Child Welfare (CEBC) is a website of evidence-based child welfare programs and practices…. Programs receive a rating based on supporting evidence, child welfare outcomes, and relevance to child welfare populations. Users can search the database by topical area, maltreatment type, scientific rating, or goals and outcomes.”

Availability: Free databaseURL:http://www.cachildwelfareclearinghouse.org/

Searching Tips:Searching by topical area or maltreatment

type recommended. The advanced search page offers a number of ways to search, however searching multiple categories dramatically reduces results, often to zero. This is primarily due to the relatively small number of recommended programs and practices in the system currently.

Manuals, Guidelines, etcNational Registry of Evidence-based Programs and Practices (NREPP)

Description: Searchable online registry of mental health and substance abuse interventions that have been reviewed and rated by independent reviewers. Intervention summaries include: descriptive information about the intervention and its targeted outcomes, quality of Research and Readiness for Dissemination ratings, a list of studies and materials submitted for review, and contact information for the intervention developer. Description based on http://www.nrepp.samhsa.gov/about.htm Accessed August 12, 2008

Availability: Free database

URL: http://www.nrepp.samhsa.gov/

Searching Tips: As of this writing, there are 106 interventions in the NREPP database. Researchers can use either the search form or browse the complete list to find

interventions.

Social Work Evidence PyramidEBP Continuum: From research-based to non research-based to promising

The following section describes the continuum of “evidence” that is the conventional guide for most mental health practitioners, researchers, and policy makers. These levels are developed from a wide range of sources and are meant to be a guide. They do not represent a single federal agency or academic center (Vandiver, 2006).

Level 1: Systematic reviews using meta-analyses or two randomized controlled clinical trials (RCT). Highest level of evidence. Where to Go: http://www.cochrane.org/reviews (The Cochrane Collaboration); http://www.campbellcollaboration.org (Campbell Collaboration - Social Welfare)

Level 2: No meta-analyses but one RCT or national consensus panel. Second highest level of evidence. Where to Go: American Journal of Psychiatry, Supplements; Ex. Practice Guidelines for the Treatment of Schizophrenia

Level 3: Quasi-experimental; uncontrolled trial or observational study; expert consensus; descriptive studies. Less rigorous research. Where to Go: http://www.psychguides.com for Expert Consensus Guidelines

Level 4:Anecdotal case reports; unsystematic clinical observations; descriptive reports; single subject designs. Non-research based but practical. Where to Go: Journal: Psychiatric Services: Case Report Column

Level 5: Clinical opinion only; studies without comparison groups; no consistently positive measured outcomes; not research based. Where to Go: Practitioner Magazine: The Family Networker

Level 6: Consistently poor outcomes for a particular population; intervention may be research based but is applied to the wrong population. Where to Go: your family lawyer for legal advice on impending lawsuit

Level 7: Emerging Best Practices or Promising Practices are those that have not been subject to rigorous research methods but that could be undergoing research review, have strong clinical utility, are liked by clients and families and are deemed culturally competent. Where to Go: http://www.modelprograms.samhsa.gov

From: http://www.ssw.pdx.edu/focus/ebp/index.php. Accessed 04/01/2009

Criminal Justice ResourcesCriminal Justice AbstractsDescription: Citations and abstracts of

international journals, books, reports, dissertations and unpublished papers on criminology and related disciplines.

Availability: Subscription database

URL:http://www.csa.com/factsheets/cja-set-

c.php

Searching Tips:DESCRIPTORS KEYWORDSNot available Evidence-based

Systematic review

Meta-analysisPractice

Guidelines

National Criminal Justice Reference Service

Description: Contains the full text of more than 150,000 publications on criminal justice, including federal, state, and local government reports, books, research reports, journal articles, and unpublished research.

Availability: Free database:

URL:http://www.ncjrs.gov/abstractdb/search.asp

Searching TipsDESCRIPTORS KEYWORDSLiterature reviews Systematic

reviewCorrections Effectiveness Meta-analysisDeterrence Effectiveness Evidence-

basedEffectiveness of Crime Randomize Prevention

controlled trialJuvenile Corrections Double-blind EffectivenessParole EffectivenessServices Effectiveness

Education ResourcesEducation Full Text

Description: Primarily a citation database (with some full text availability), it indexes articles from ~570 English-language periodicals and yearbooks published in the United States and elsewhere.

Availability: Subscription database:

URL:http://www.hwwilson.com/databases/

educat.htm

Searching Tips

DESCRIPTORS:Meta-analysis of researchClinical trials

Educational Resources Information Clearinghouse (ERIC)

Description: Indexes research reports and journal literature in the field of education from 1966 to the present. Includes all educational levels and all aspects of education in all disciplines.

Availability:Subscription database: available on many

commercial vendor systems Free database:

http://www.eric.ed.gov/ERICWebPortal/Home.portal?_nfpb=true&_pageLabel=basic&Clearme=true

Searching TipsDESCRIPTORS:Literature reviewsMeta AnalysisInstructional EffectivenessCounseling EffectivenessOutcomes of TreatmentGuidelines

Manuals, Guidelines, etcWhat Works Clearinghouse

Description: “The WWC aims to promote informed education decision making through a set of easily accessible databases and user-friendly reports that provide education consumers with high-quality reviews of the effectiveness of replicable educational interventions (programs, products, practices, and policies) that intend to improve student outcomes.” (Description from: http://ies.ed.gov/ncee/wwc/overview/ Accessed July 10, 2008)

Availability: Free web-based resource

URL: http://ies.ed.gov/ncee/wwc/

Searching Tips: Primarily arranged by intervention topic area (e.g., early childhood education, dropout prevention, middle school math, etc): list all topic interventions, Keyword searching within topic, or sort interventions using predefined criteria.

Web Search EnginesGoogle ScholarDescription: Google Scholar extracts records from Medline, ERIC, and other freely

accessible databases in addition to institutional repositories and other high value academic and research websites.

Availability: Free search engineURL: http://scholar.google.comSearching Tips: Use high value evidence-based practice keywords (systematic review,

meta-analysis, etc) for better search results.

ScirusDescription: “…the most comprehensive scientific research tool on the web. With over

450 million scientific items indexed at last count, it allows researchers to search for not only journal content but also scientists' homepages, courseware, pre-print server material, patents and institutional repository and website information.” http://www.scirus.com/. Accessed August 13, 2008.

Availability: Free search engineURL: http://www.scirus.com/srsapp/Searching Tips: Strengths lie more in the medicine and mental health, with less depth in

some applied social science disciplines such as criminology and education.

Use high value evidence-based practice keywords (systematic review, meta-analysis, etc) for better search results. Results page offers additional limits by content sources (BioMed Central, ScienceDirect, etc), file types (PDF, HTML, Word), and helpful ‘refine your search’ suggestions.