Rigor AND Relevance
Kurt C. Stange, MD, PhDEditor, Annals of Family Medicine
Gertrude Donnelly Hess, MD Professor of Oncology Research
Professor of Family Medicine, Epidemiology & Biostatistics,
Sociology and OncologyCase Western Reserve University
Overview• Reflections on Ornstein & Ruhe presentations
• The problem
• An integrative epistemology
• A transdisciplinary whole systems approach
• QI “laboratories” - PBRNs
• Participatory approaches to both R & D
• Mixed methods – quantitative & qualitative
• A complexity science underpinning
• Putting it together
Reflections on Presentations by Ornstein & Ruhe
• Responding to common stimuli
• Intervention approaches
• Evaluation Methods
Responding to Common Stimuli• Real-world primary care
– Incompatible reimbursement/business model– Competing demands / opportunities– PBRNs
• Diversity– Undesirable (low rates of effective practices)– Desirable (local adaptation)
• Melding research & quality improvement– Resistance to randomization
• Peer review / categorical funding
Intervention Approaches
• Multifaceted– Multiple processes, tools & outcomes– Moving toward multilevel Chronic Care Model
• Individualization of shared best practices– Outside facilitation / consultation– Shared learning
• Complexity science principles
Evaluation Approaches• Mixed methods designs
– Group RCT → pre/post design– Concurrent qualitative process evaluation
• Tailoring of intervention
• Measures– Quantitative process & outcome
• Inductive approach– Observation, interviews– Discovery of what works locally
The Problem• Rigor vs. relevance
• Internal vs. external validity
• Isolation of a phenomenon from context (so that it can be more rigorously studied) when context matters
• Our methods, theories, world view don’t match the problems or apparent solutions
4 Ways of Knowing
Adapted from:
Wilber, K. Sex, Ecology, Spirituality. 1995/2000, Boston: Shambhala Publications, Inc.
Wilber, K. A Brief History of Everything. 1996, Boston: Shambhala Publications, Inc.
Inner Reality
Outer Reality
Individual “I” “It”
Collective “We” “It”
4 Ways of Knowing About Health & Health Care
Adapted from:Stange KC, Miller WL, McWhinney I. Developing the knowledge base of family practice. Fam Med. 2001; 33(4):286-297.
1Clinician, Patient,
Worker, Policymaker
4Disease,
Treatment
2Family, Practice,
Community
3Systems,
Organization
4 Ways of Knowing About Health & Health Care
Borders 1-2 Relationship2-3 Justice3-4 Prioritization4-1 Information mastery1-3 Collaboration2-4 Illness
1Clinician, patient,
worker, policymaker
4Disease,
Treatment
2Family, Practice,
Community
3Systems,
Organization
Multiple Ways of Knowing
SYNTHESIS of ways of knowingThe craft of policy & general practice is the integration and application of knowledge of biomedicine, health care systems, individuals, families, communities & self.
1-2 Relationship2-3 Justice3-4 Prioritization4-1 Information mastery1-3 Collaboration2-4 Illness
1Clinician, patient,
worker, policymaker
4Disease,
Treatment
2Family, Practice,
Community
3Systems,
Organization
BORDERS among ways of knowing
Multiple Ways of Knowing
1 INFORMATION 4 (Quadrant 1) MASTERY (Quadrant 4)
CLINICIAN Evidence-Based DISEASESelf-Awareness Medicine ScienceReflection, Journaling Learning Epidemiology &
Experimentation
RELATIONSHIP INTEGRATION PRIORITIZATIONHuman Interactions Health Care & Healing ValueParticipant Observation Transdisciplinary, Cost-Effectiveness Analysis
Multi-method,Participatory
(Quadrant 2)
PATIENT, (Quadrant 3)
FAMILY SYSTEMCOMMUNITY OrganizationsPersonal Values Health ServicesIn-Depth Interviews JUSTICE ResearchLiving in Place Social Values
Policy Analysis2 3
For each item, bold capitalized words on the first line signify "FOCUS OF KNOWLEDGE," normal text on thesecond line signifies "Task of Understanding," and italicized words on the third line signify "Mode of Inquiry."
Honoring Different Ways of Knowing
• Acknowledging different perspectives– In planning studies– In interpreting findings
• Multiple groups working separately but communicating
• Paying attention to (or at least considering) other ways of knowing, even if only working on one.
Integrating Different Ways of Knowing
• Sequential studies
• Simultaneous studies– Multiple viewpoints– Multiple methods
• Studies of the border regions
• Studies of the integrative function
Transdisciplinary, Whole Systems Collaboration
can create abundance by sharing scarcity.
Crabtree BF, Miller WL, Adison RB, Gilchrist VJ, Kuzel A. Exploring Collaborative Research in Primary Care. Thousand Oaks, California: Sage Publications; 1994.
Leadership for Collaborative Research, Development
and Quality Improvement
• Whole system– Top down– Bottom-up
Thomas P, Griffiths F, Kai J, O'Dwyer A. Networks for research in primary health care. BMJ. 2001;322:588-590.
Thomas P. Integrative Primary Health Care: Leading, Managing, Facilitation. London: Radcliff Press, 2006.
A Typology of Collaboration
• Multidisciplinary
• Interdisciplinary
• Transdisciplinary
Crabtree BF, Miller WL, Adison RB, Gilchrist VJ, Kuzel A. Exploring Collaborative Research in Primary Care. Thousand Oaks, California: Sage Publications; 1994.
Multidisciplinary Research
• Multiple disciplines
• Each contributes their piece to solving a problem
• Like an edited book or separate presentations by multiple experts
Interdisciplinary Research
• A conversation between and among disciplines
• Working together on solving a common problem
• Like a collaborative health care team
Transdisciplinary Research
• A sustained conversation across and beyond disciplinary boundaries
• Creates a new shared language
• Such as the emergence of family systems medicine
6 Stages of Collaboration
• Acceptance / validation
• Shared expectations
• Declaring group process
• Action consensus
• Common space
• Sustained common action
Barriers to Collaborative Process
• Rhetorical stones– Power heaping– Shaming– Jargon hurling
• Powerful hegemony– Rationality vs. / and – Wonder, confession gratitude, receptivity to gift and mystery
• Tension between pragmatism and reflection
• Tension between individual & systems focus
Tactics for Advancing the Process
• Brainstorming
• Humor
• Storytelling
• Silence and time out for play
The Actual Process
• Non-linear
• Blurring and blending of levels
• Back and forth between levels
• Destabilized by time pressure
• Enhanced by flexibility, tact, patience and persistence
Dangers of Collaborative Research Relationships
• Sloppiness from training down
• Suppression of individuality
• Cultism
• Political nature of groups
• Minimizing these requires self-reflection and challenging
6 Ingredients for Successful Collaborations
• Linkage perspective
• Local context and action
• Problem-focused
• Appropriate methods
• Critical multiplism
• Coordination by a generalist researcher
Leadership for Collaborative Research
• In the face of change and uncertainty– Animation
• Provide initial direction• Encourages updating• Facilitates respectful interaction - trust, trustworthiness
– Improvisation• A hunch held lightly is a direction to be followed, not a
decision to be defended
– Lightness• “I don’t know”
– Authentication• Sensemaking
– Learning
Weick KE. Leadership as the legitimation of doubt. In Bennis W, Spreitzer GM, Cummings TG, (eds). The Future of Leadership. San Francisco: Jossey-Bass, 2001.
R &D &QI “Laboratories”
• Whole systems e.g. HMO Research Network
• NIH Research Center Model e.g. Cancer Centers
• Practice-based research networks (PBRNs)
PBRNs• Affiliated practices devoted to patient care
• Often academic or other partners
• Engage frontlines wisdom to – Develop or frame questions– Gather data– Interpret findings– Implement findings
• More generalizable patient populations, & theoretically, more transportable research
http://ahrq.gov/research/pbrnfact.htm
Primary Care PBRNs• Laboratories for primary care research
• 111 networks in 44 states
• Translate research into practice
• Translate practice into research
Nutting P, Beasley J, Werner J. Practice-based research networks answer primary care questions. JAMA. 1999;281:686-688.
Thomas P, Griffiths F, Kai J, O'Dwyer A. Networks for research in primary health care. BMJ. 2001;322:588-590.
Lanier D. Primary care practice-based research comes of age in the United States. Ann Fam Med. 2005; 3(suppl 1):S2-S4.
Primary Care PBRNs• Answer important questions
• Challenging current environment
• With support, ready for prime-time
• Answer to the call of the NIH Roadmap
Contemporary Challenges for Practice-Based Research Networks. Ann Fam Med; 2005; 3 (suppl 1). http://www.annfammed.org/content/vol3/suppl_1/index.shtml
Nutting PA, Stange KC, eds. Prescription for Health: Changing Primary Care Practice to Foster Healthy Behaviors. Ann Fam Med; 2005; 3 (suppl 2). Full text free at: http://annfammed.org/content/vol3/suppl_2/index.shtml
Green LA, Dovey SM. Practice based primary care research networks. They work and are ready for full development and support. BMJ.2001;322:567-568.
Nutting PA, Beasley J, Werner JJ. Practice-based research networks answer primary care questions. JAMA. 1999;281:686-688.
1000 persons
800 report symptoms
327 consider seeking medical care
217 visit a physician’s office (113 visit a primary care physician’s
office) (PBRN Research)
65 visit a complementary or alternative medical care provider
21 visit a hospital outpatient clinic
14 receive home health care
13 visit an emergency dept
8 are hospitalized
<1 is hospitalized in an academic medical center (Most Research)
Fig. Results of a reanalysis of the monthly prevalence of illness in the community and the roles of various sources of health care. (Green LA et al., N Engl J Med 2001, 344:2021-2024)
The Ecology of Medical Care
Community Participatory Research
• Knowledge, resources, involvement of communities are key to success of research and its transportability & sustainability
• 3 primary features– Collaboration– Mutual education– Acting on results developed from research
questions relevant to the community
Macaulay AC, Commanda LE, Freeman WL, et al. Participatory research maximises community and lay involvement. BMJ. 1999;319:774-778.
Facilitation of Participatory Research
• Continuity
• Timeliness
• Flexibility
• Clear and explicit expectations
• Appropriate & instructive accountability
• A vision for participatory research
• Tailored technical assistance
• Enhancing partner collaboration & support
Mercer SL, MacDonald G, Green LW. Participatory research and evaluation: from best practices for all states to achievable practices within each state in the context of the Master Settlement Agreement. Health Promot Pract. 2004;(3 Suppl):167S-178S.
Multimethod Approaches
• Increasingly accepted
• Efficient for generating new knowledge
Stange KC, Zyzanski SJ. Integrating qualitative and quantitative research methods. Fam Med, 1989; 21:448-451.
Stange KC, Miller WL, Crabtree BF, O’Connor PJ, Zyzanski SJ. Multimethod research: Approaches for integrating qualitative and quantitative methods. J Gen Intern Med. 1994; 9:278-282.
Crabtree BF, Miller WL. Doing Qualitative Research. 2nd Ed. Thousand Oaks, California: Sage Publications, 1999.
Crabtree BF, Miller WL, Stange KC. Understanding practice from the ground up. J Fam Pract, 2001; 50:881-887.
Borkan JM. Mixed methods studies: a foundation for primary care research. Ann Fam Med. 2004;2(1):4-6
NCI Conference on Mixed Methods Research, July, 2004, Denver, CO.
Multimethod Research • Quantitative methods
• Counting descriptions• Testing a priori hypotheses• Seek to isolate phenomenon from context
• Qualitative methods• Rich descriptions• Discovery; testing evolving hypotheses• Seek to understand meaning and context
• Integrated use• Qualitative, then quantitative• Quantitative, then qualitative• Simultaneous
Stange KC, Miller WL, Crabtree BF, O’Connor PJ, Zyzanski SJ. Multimethod research: Approaches for integrating qualitative and quantitative methods. J Gen Intern Med. 1994; 9:278-282.
Mixed methods allow you to have your cake and eat it too.
Additional Theoretical Underpinnings
• Complexity science principles
• Chronic care model
• Re-Aim
• Framework for complex interventions
• Gree nhalgh
Practices as Complex Adaptive Systems
• Complex behavior emerges from relationships among agents
• Simple rules
• Recurrent patterns
• Co-evolution
• Dependence on initial conditions
• Non-linearity
• Strategies for intervention• Joining• Transforming• Learning
Miller WL, Crabtree BF, McDaniel RA, Stange KC. Understanding primary care practice: A complexity model of change. J Fam Pract, 1998; 46:369-376.
Miller WL, McDaniel RA, Crabtree BF, Stange KC. Practice Jazz: Understanding variation in family practices using complexity science. J Fam Pract, 2001; 50:872-878.
Stroebel CK, McDaniel RR Jr, Crabtree BF, Miller WL, Nutting PA, Stange KC. Using complexity science to inform a reflective practice improvement process. Jt Comm J Qual Patient Saf, 2005; 31:438-446.
Three Key InsightsThree Key Insights
• Practices are co-creative participants in dynamic fitness landscapes.
• There are multiple ways to achieve effective health care delivery in practice.
• The best strategies for improving practice pay attention to improving relationships among stakeholders.
McDaniel RA, 2005
Chronic Care Model• Community• Health care system• Resources, policies & organization
– Self-management support– Delivery system design– Decision support– Clinical information systems
• Interaction– Informed, activated patient– Prepared, proactive practice team
Wagner EH. Chronic disease management: What will it take to improve care for chronic illness: Effective Clinical Practice. 1998;1:2-4.
Glasgow RE, Orleans CT, Wagner EH, Curry SJ, Solberg LI. Does the chronic care model serve also as a template for improving prevention? Millbank Q. 2001;79:579-612.
Re-Aim• Reach• Efficacy/effectiveness• Adoption• Implementation• Maintenance
www.re-aim.org
Glasgow RE, McKay HG, Piette JD, Reynolds KD. The RE-AIM framework for evaluating interventions: what can it tell us about approaches to chronic illness management? Patient Educ Couns. 2001;44:119-127.
Glasgow R, Magid D, Beck A, Ritzwoller D, Estabrooks P. Practical clinical trials for translating research to practice: design and measurement recommendations. Med Care. 2005;43:551-557.
Design & Evaluation of Complex Interventions
Campbell M, Fitzpatrick R, Haines A, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):694-696.
Next Generation of Diffusion of Health Service Innovations
• Theory-driven• Process rather than ‘package’ oriented• Ecological• Addressed with common definitions, measures, tools• Collaborative & coordinated• Multidisciplinary & multimethod• Meticulously detailed• Participatory
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581-629.
An Incremental Approach
• Work on or pay attention to multiple levels
• Pursue research, development & shared learning
• Develop participatory relationships that transcend single projects
• Integrate qualitative & quantitative methods
• Reconsider enabling the current dysfunctional system versus fostering its transformation
Top Related